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ABSTRACT 


A three-dimensional 

homomorphic 

model  of  human 

color  v i s i 

i on 

based  on 

neurophys i o 1 

ogical  and 

psychophys i ca 1 

ev i dence 

i s 

presented. 

This  model 

permi ts  the 

quant i tat i ve 

de  f i n i t i on 

of 

percoptua I I y important  parameters  6uch  as  brightness,  saturat  ion, 
hue  and  strength.  By  modelling  neural  interaction  in  the  human 
visual  system  as  three  linear  filters  operating  on  perceptual 
quantities,  this  model  accounts  for  the  automatic  gain  control 
properties  of  the  eye  and  for  brightness  and  color  contrast  effects. 

In  relation  to  color  contrast  effects,  va  psychophysical 

i f i ' " ? 

experiment  was  performed^  . It  utilized"  a high  quality  color 
television  monitor  driven  by  a general  purpose  digital  computer. 
This  experiment,  based  on  the  cancellation  by  human  subjects  of 
simultaneous  color  contrast  illusions,  allowed  vtbsf'  measurement  of 
the  low  spatial  frequency  part  of  the  frequency  responses  of  the 
filters  operating  on  the  two  chromatic  channels  of  the  human  visual 
system.'  The  experiment  is  described  and  its  results  are  discussed. 

Next,  tho  model  is  6houn  to  provide  a suitable  framework  in 
which  to  perform  digital  images  processing  tasks.  First, 
applications  to  color  image  enhancement  are  presented  and  discussed 
in  relation  to  photographic  masking  techniques  and  to  the  handling 
of  digital  color  images.  Second,  application  of  the  model  to  the 
definition  of  a distortion  measure  between  color  images  (in  the 


V 


cionoo  of  Shannon's  rate-distortion  theory),  meaningful  in  te'-ms  of 
human  evaluation,  is  shown.  Mathemat i ca I norms  in  the  "perceptual" 
apace  defined  by  the  model  are  used  to  evaluate  quantitatively  the 
amount  of  subjective  distortion  present  in  artificially  distorted 
color  images.  Third,  applications  to  image  transmission  and  coding 
are  presented.  Results  of  a coding  experiment  yielding  digital 
color  images  coded  at  an  average  bit  rate  of  1 bit/pixel  are  shown. 

Finally  conclusions  are  drawn  about  the  implications  of  this 
research  from  the  standpoint  of  psychophysics  and  of  digital  image 
procosG i ng. 


vi 
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CHAPTER  1 


PSYCHOPHYSICAL  AND  NEUROPHYSIOLOGICAL  BACKGROUND 


1.1  Colorimetry 

The  main  fact  about  color  vision  is  its  basic  trichromatic 
nature.  Overwhelming  support  has  been  provided  over  the  past  one 
hundred  and  fifty  years  for  the  notion  first  put  forth  clearly  by 
Thomas  Young  that  three  separate  varieties  of  receptors  underlie 
color  vision. 

In  particular  the  experimental  laws  of  color-matching  are  an 
indirect  verification  of  this  fact.  They  state  that  over  a wide 
range  of  conditions  of  observation  colors  can  be  matched  by  additive 
mixtures  of  three  fixed  primary  colors.  These  primaries  must  be 
ouch  that  none  of  them  can  be  matched  by  a mixture  of  the  other  two. 
Furthermore  those  matches  are  linear  over  a wide  range  of  observing 
conditions  which  implies  two  things.  First,  the  match  between  any 
two  colors  continues  to  hold  if  the  correspond  i ng  stimuli  are 
increased  or  reduced  by  the  same  amount,  their  respective  relative 
spectral  energy  distributions  being  unchanged  (scalability 
condition).  Second,  if  colors  C and  0 match  and  colors  E and  F 
mq^ch.  thlfr  the  additive  mixture  of  colors 
correspond i ng  additive  mixture  of  colors  D and 

This  linearity  property  allows  us  to 
three-dimensional  vectors  and  color  matches 


C and  E matches  the 
F. 

represent  colors  by 
by  linear  equations 
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Stockham  [53,  51]  , was  the  first  one  to  suggest  that  a 
multiplicative  rather  than  linear  system  was  better  fitted  to  the 
structure  of  black  and  white  images  and  furthermore  was  simi  I ar  to 
the  processing  occuring  in  the  earlier  stages  of  the  human  visual 
path.  His  efforts  to  define  and  use  a multiplicative  homomorphic 
visual  model  have  been  carried  on  and  fully  justified  by  Baudelaire 
[1]  and  extended  by  Baxter  C2]  . The  use  of  the  model  for  the 
definition  of  a subjective  distortion  measure  has  been  explored  by 
Mannos  [33]  and  Rom  [39] . 

Very  briefly,  the  multiplicative  homomorphic  model  has  the 
following  property:  by  modelling  the  nonlinearity  in  the  absorption 
of  light  energy  in  the  retina  as  well  as  neural  interaction  later  on 
the  visual  path,  it  accounts  for  brightness  contrast  and  brightness 
constancy  and  realizes  the  separation  of  reflectance  from 
i I I urn i na  t i on. 

Effects  similar  to  thos  p'  /iously  mentioned  are  also  present 
in  color  vision.  Color  constancy  effects  which  show  the  tremendous 
ability  of  the  eye  to  adapt  instantaneously  to  ambiant  illumination 
even  if  heavily  color  biased  are  well  known  [6,  32].  Color  contrast 
effects  and  color  Mach  Bands  have  also  been  recognized  [22,  28]. 

But  a clear  formalization  of  these  phenomena  is  hampered  by  the 
three-dimensionality  of  color  vision.  A color  can  be  defined  in 
terms  of  its  brightness,  hue  and  saturation  and  interaction  between 
colors  can  cause  the  change  of  all  these  parameters.  Clearly,  a 
model  of  color  adaptation  and  a quantitative  description  of  such 
parameters  as  color  brightness  and  saturation  would  be  welcome. 

Knowing  this,  it  seems  very  appealing  to  attempt  to  model  human 
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color  vision  using  a multiplicative  homomorphic  model.  This 
approach  should  guarantee  that  the  new  model  reduces  to  the  old  one 
for  achromatic  vision.  Also,  it  would  allow  us  to  use  the  powerful 
tool  of  Fourier  analysis  which  has  proven  to  be  well  suited  to  a 
precise  description  of  achromatic  brightness  contrast  and  constancy 
effects  [1,  6]  and  is  therefore  hoped  to  perform  as  well  for  color 
of  facts. 

In  our  opinion,  the  achievements  of  this  research  are  twofold: 
On  the  side  of  psychophysics,  we  have  shown  that  by  using  a model 
based  on  neurophysiological  and  psychophysical  evidence  as  a 
theoretical  background,  and  a high  precision  television  monitor 
driven  by  a digital  computer  as  a tool,  we  could  obtain  some 
relevant  experimental  results  about  the  way  chromatic  information  is 
processed  by  the  human  visual  system. 

On  the  side  of  image  processing,  we  have  shown  that  this  model 
allowed  us  to  think  about  and  perform  in  a meaningful  way  our  image 
processing  tasks. 

Chapter  1 lays  the  psychophysical  and  neurophysiological  basis 
which  is  used  in  chapter  2 to  develop  a quantitative  description  of 
achromatic  and  chromatic  information  processing  in  the  visual  path. 
Chapter  3 describes  the  psychophysical  experiment  by  which  we  probed 
the  chromatic  channels.  Chapters  4 and  5 are  dedicated  to  the 
applications  of  the  model  to  image  processing.  Chapter  4 describes 
some  ways  of  enhancing  co I or  images.  Chapter  5 elaborates  on  the 
use  of  the  model  to  define  a perceptual  distance  between  color 
images  and  on  related  problems  of  transmission  and  coding.  Chapter 
G draws  the  conclusions  of  this  research  project. 


INTRODUCTION 


The  research  presented  in  thi6  report  is  concerned  with  image 
processing.  In  a very  general  sense,  image  processing  can  be 
thought  of  as  a transf  ormat  i on  of  an  image  from  one  form  into 
another.  The  results  can  appear  as  another  image,  a set  of 

decisions,  a model  or  parametr i zat i on.  Here,  we  will  restrict 
ourselves  to  the  case  where  the  transformation  yields  as  its  output 
another  image  which  is  different  from  the  original  in  some  desirable 
way. 

Since  a human  observer  is  likely  to  be  the  last  element  in  the 
processing  chain,  it  seems  natural  to  become  interested  in  the 
properties  of  the  human  visual  system.  The  idea  of  applying  the 
current  knowledge  about  the  human  visual  system  to  the  field  of 
image  processing  is  a recent  one  though.  It  stemmed  from  the 
availability  of  digital  methods  on  one  hand  and  the  lack  of  a 
quant i tat i ve  definition  of  subjective  image  quality  on  the  other 
hand. 

Indeed,  digital  methods  are  character i zed  by  their  flexibility 
and  their  precision.  Good  subjective  distortion  measures  built  from 
a clear  and  concise  formalism  have  begun  to  emerge  for  achromatic 
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between  such  vectors.  If  C represents  a given  color  and  R,  G and  B 
roprescnt  unit  amounts  of  three  fixed  primaries  then  the  equation 
C-RR+GG+BB 

expresses  the  fact  that  the  given  color  is  matched  by  an  additive 
mixture  of  quantities  R,  G and  B respectively  of  the  given 
primaries.  R,  G and  B are  called  the  tristimulus  values  of  the 
given  color  in  the  particular  6et  of  primaries  used.  Another  set  of 
frequently  used  quantities  i6  the  chromaticity  coordinates  r,  g,  b 
defined  by 

r - R/  (R+G+B) 
g ■ G/  (R+G+B ) 
b « B/ (R+G+B) 

The  linearity  property  also  allous  us  to  define  completely  the 
color-matching  properties  of  the  observing  eye  in  the  given  primary 
system  by  three  functions  of  wavelength  r(A),  g(M  and  b (IX)  called 
color-matching  functions.  If  the  color  C has  a spectral  energy 
distribution  p(X)  then  its  tristimulus  values  are 
R-X  p(A)r(A)dA 
G-X  p(MyO>)dX 
B-X  p(A)b(X)dX 


where  the 

integrals  are  taken  over  the  visib 

le  spectrum.  Colors  C, 

and  C,  of 

spectral  energy  distributions  p,  (X) 

and  p,(X)  match  if  and 

only  if 

X p,(A)r(A)  dA«X  p,(A)r(A)dX 

(1.1a) 

X p,  (A)  g (X)  dX«  J p,(X)g(X)dX 

(1.1b) 

X p,(X)b(X)dX-X  p,(X)b(X)dX 

(1.1c) 
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A set  of  such  color-matching  functions  is  shown  in  figure  1.1. 

The  linearity  property  also  allows  us  to  describe  a change  of 
primaries  as  a simple  matrix  multiplication  operation  whereby  the 
new  tri  stimulus  values  L,  H and  S of  a given  color  and  the  new 
color-matching  functions  1(A),  m (*)  and  s (X)  are  given  by 

Tn  " A ?0  tn  “ At0 

where  Cl,M.S]\  ?0  - CR,G,B]\  tn  “ 1 1 U)  , m (X) , s U)  ] \ 

t0  - Ir  (X)  , g DO  , b i\)  ] * and  A is  a three  by  three  matrix  whose  column 
vectors  are  the  tri  stimulus  values  of  the  old  primaries  in  the 
system  of  new  primaries. 

A standard  observer  has  been  developed  in  1931  by  the  CIE 
( Commission  Internationale  de  I ' Ec I a i rage" ) whose  color-matching 
functions  are  denoted  by  x (A) , y(>J  and  z(A)  and  shown  in  figure 
1.2.  The  matches  predicted  with  their  aid  using  equations  1.1a  to 
1.1c  represent  average  matches  for  humans  with  normal  color  vision 
as  opposed  to  humans  with  any  one  of  the  several  forms  of  color 
blind  vision. 

1.2  Spectral  sensitivities  of  the  cone  receptors 

The  physiological  basis  of  trichromatic  color  matches  is  the 
linear  absorption  of  light  at  photopic  levels  of  illumination  (when 
the  effect  of  rods  receptors  is  negligible)  by  three  types  of  cone 
receptors  in  the  human  fovea.  However,  specification  of  the 
spectral  absorption  character i st i cs  of  the  three  different  cone 
types  has  been  difficult  since  it  has  not  been  possible  to  extract 
cone  pigments  from  the  eyes  of  primates  112).  A number  of  different 
approaches  has  been  taken  to  yield  this  information  which  is 
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essential  to  an  understanding  of  color  vision.  Psychophysical 
evidence  has  come  from  Willmer  [60] , Blackwell  and  Blackwell  [3], 
Stiles  [43]  and  Pitt  [36]  among  many.  Rushton  [40.  41]  has 
attempted  to  answer  the  same  question  by  reflection  densitometry. 
Attempts  have  also  been  made  to  measure  those  curves  from  unit 
recordings  later  in  the  visual  pathway  (De  Valois  1365,  [11],  Uliesel 
and  Hubei  1366,  [53]). 

All  these  s.tudies  agree  well  on  two  points.  The  first  is  that 
thero  are  three  types  of  cones  with  their  absorption  curves  peaking 
at  about  445,  540  and  570  nm.  The  second  is  that  these  absorption 
curves  are  quite  broad  and  fit  to  a first  approximation  the  Dartnall 
nomogram  (Dartnall  1353,  [7]). 

The  common  reference  to  those  receptors  as  "red",  "green"  and 
"blue"  receptors  stems  from  the  spread  of  Young-He I mho  I tz  type  of 
theories  of  color-vision.  This  terminology  is  confusing.  For 
example,  let  us  consider  the  case  of  the  570  pigment  cone  so 
misleadingly  called  by  many  the  "red  receptor".  Not  only  is  it  not 
limited  in  its  sensitivity  to  the  part  of  the  spectrum  we  see  as  red 
(about  600  to  700  nm)  but  it  is  not  even  maximally  sensitive  in  this 
spectral  region.  In  fact  it  is  more  sensitive  to  that  part  of  the 
spectrum  we  see  as  pure  green  (510  nm)  than  to  pure  red  (650  nm)  ! 

So  both  on  account  of  its  point  of  maximal  absorption  and  of 
the  breadth  of  its  absorption  curve  it  is  a gross  misnomer  to  call 
the  570  receptor  a "red  receptor"  or  the  540  cone  a "green 
rocoptor".  But  worst  is  the  fact  that  calling  those  receptors  "red" 
or  "green"  implies  that  their  excitation  signals  "red"  or  "green" 
further  away  on  the  visual  path. 
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We  ui I I use  the  terms  L-cones  (containing  the  long-wavelength 
peaking  pigment),  fl-cones  (containing  the  medium-wavelength  peaking 
pigment)  and  S-cones  (containing  the  short-wavelength  peaking 
pigment)  to  denote  the  cone  types  containing  the  570,  540  and  445  nm 
peaking  pigments. 

1.3  Nonlinear  response  of  the  cones 

Physiological  studies  have  shown  that  the  activity  or  response 
of  the  cone  receptors  is  proportional  not  to  the  intensity  of  the 
'Stimulus  but  rather  to  its  logarithm  over  a large  range  of  stimulus 
intensities  (54,  37,  6) . One  psychophysical  equivalent  of  these 
physiological  data  is  the  well  known  Fechner’s  law  which  states  that 
the  just  noticeable  brightness  difference  is  proportional  to  the 
logarithm  of  the  stimulus  intensity  (27). 

It  i6  interesting  to  point  out  that  this  quae  i - I ogar  i thm  i c 
mapping  of  light  intensity  into  neural  activity  at  the  output  of  the 
three  cones  allows  the  retina  to  accept  inputs  over  a very  wide 
range  of  intensities  (24).  It  has  already  been  recognized  (58)  and 
it  will  be  shown  quantitatively  in  this  report  that  this  ability  for 
visual  stimuli  compression  at  the  receptor  level  together  with 
noural  interaction  later  on  the  visual  path  is  the  fundamental  basis 
for  visual  adaptation. 


1.4  The  processing  of  brightness  and  color  information  past  the 
retina 


Wo  know  that  in  higher  vertebrates  the  retina  projects  itself 
to  the  lateral  geniculate  nucleus  (LGN)  and  from  there  to  the 
striate  cortex  (12).  Unfortunately  it  is  far  from  possible  at  our 


present  state  of  knowledge  to  describe  the  central  neural  processing 
of  visual  information  past  the  LGN.  However  from  the  close 
parallels  one  finds  between  the  responses  of  cells  at  the  LGN  level 
and  color  perception,  it  would  appear  that  much  of  the  processing  of 
color  information  takes  place  before  the  cortex,  With  the  risk  of 
oversimplifying  we  will  summarize  the  results  obtained  by 
neurophysiological  experiments  on  LGN  cells  of  primates  118,  13, 
53]  . 

Four  varieties  of  so-called  spectrally  opponent  cells  were 
found  in  the  LGN.  They  showed  excitation  to  some  wavelengths  and 
inhibition  to  others  when  the  eye  was  stimulated  with  flashes  of 
monochromatic  lights.  Moreover  this  type  of  response  was  maintained 
over  wide  intensity  ranges  thus  indicating  that  the  cells  responded 
differentially  to  color  rather  than  to  luminance  differences.  The 
colls  whose  responses  showed  maximum  excitation  around  500  nm  and 
maximum  inhibition  around  G30  nm  were  called  green  excitatory,  red 
inhibitory  cells  (+G-R).  Other  cells  showing  rough  mirror  image 
responses  were  called  red  excitatory,  graen  inhibitory  cells  (+R-G) . 
The  cells  showing  maximum  excitation  (inhibition)  at  600  nm  and 
maximum  inhibition  (excitation)  at  440  nm  were  named  yellow 
excitatory,  blue  inhibitory  cells  (+Y-B)  and  blue  excitatory,  yellow 
inhibitory  cells  (+B-Y)  respectively. 

In  addition  to  these  four  varieties  of  spectrally  opponent 
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vision  and  were  named  white  excitatory,  black  inhibitory  cells 
(+Wh-Bl)  and  black  excitatory,  white  inhibitory  cells  (+BI-Wh). 

Furthermore  these  studies  were  able  to  answer  the  question  of 
what  are  the  cone  inputs  to  the  LGN  cells.  Red-green  cells  (+R-G 
and  +G-R)  receive  their  inputs  from  the  L and  M-cones.  One  of  the 
cone  type  is  excitatory,  the  other  inhibitory.  Ulhich  cones  feed 

into  the  yellow-blue  system  is  a matter  of  dispute.  It  is  certain 
that  they  receive  inputs  from  the  S-cones  but  whether  the  other 
inputs  are  from  the  L or  M-cones  (or  possibly  both)  is  uncertain. 
Since  the  weight  of  the  evidence  seems  to  indicate  that  the  first 
situation  is  the  one  which  actually  exists  [121,  we  will  assume  that 
the  second  type  of  inputs  is  received  from  the  L-cones.  Like  the 

rod-green  cells,  one  cone  input  to  the  yellow-blue  cells  is 
excitatory,  the  other  inhibitory.  For  the  non-opponent  cells  there 
is  good  evidence  that  both  the  L and  M-cones  (and  possibly  S-cones 
as  well)  are  feeding  into  them  and  acting  in  the  same  direction. 

There  is  thus  much  evidence  from  unit  recording  experiments  in 
favor  of  the  existence  of  separate  chromatic  and  achromatic  channels 
each  gaining  its  information  from  the  same  receptors  but  processing 
it  in  different  ways.  Spectrally  non-opponent  cells  which 
correspond  to  the  achromatic  channel  add  together  the  (Log)  outputs 
of  the  L,  M and  S-cones.  Spectrally  opponent  cells  which  correspond 
to  the  chromatic  channels  subtract  the  (Log)  outputs  of  the  L and 
M-cones  for  the  red-green  system,  of  the  L and  S-cones  for  the 
yellow-blue  system. 

It  i 6 clear  that,  although  the  physiological  processes  involved 
are  quite  different  from  those  Hering  C25)  proposed  in  1878,  the 
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coll  typos  found  in  the  LGN  correspond  very  closely  to  the  types  of 
retinal  outputs  he  hypothesized  to  account  for  color  appearances. 

1.5  Spatial  organization  of  LGN  cells 

The  cone  inputs  to  the  LGN  cells  are  spatially  distributed. 
The  spectrally  non-opponent  cells  have  a center-surround 
organization  where  the  center  is  excitatory  and  the  surround 
inhibitory  or  vice  versa  [59].  With  regard  to  the  spectrally 
opponent  cells,  the  situation  is  somewhat  more  complicated.  Two 
types  of  organization  are  found.  The  most  common  variety  has  an 
excitatory  input  from  one  cone  type  in  the  center  and  an  inhibitory 
input  from  the  other  type  in  the  surround  or  vice  versa.  By  far  the 
largest  number  of  these  cells  are  red-green  cells  with  only  a few 
yellow-blue  cells.  The  second  variety  is  found  much  less  frequently 
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It  is  interesting  to  investigate  whether  this  spatial 
organization  can  account  for  spatial  effects.  The  color  seen  at  one 
point  in  space  at  a particular  time  does  not  depend  only  upon  the 
stimulus  at  that  point  in  space  and  time  but  also  on  the  stimuli  at 
neighboring  points.  One  very  well  known  spatial  interaction  is 
called  contrast.  Simultaneous  contrast  refers  to  the  fact  that  a 
surround  of  one  color  or  brightness  tends  to  induce  the  opposite 
color  or  brightness  into  a neighboring  region.  For  example,  in 
simultaneous  brightness  contrast,  a grey  appears  black  in  a white 
surround  and  white  in  a black  surround.  In  simultaneous  color 
contrast,  a grey  appears  reddish  in  a green  surround,  greenish  in  a 
red  surround. 

It  is  commonly  supposed  that  the  center-surround  organization 
of  non-opponent  cells  in  the  LGN  accounts  for  simultaneous 
brightness  contrast  although  this  has  been  recently  questioned  [14], 
Gn  the  other  hand  the  spatial  organization  of  opponent  cells 
predicts  the  absence  of  pure  color  Mach  Bands  (induced  by  constant 
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This  is  to  be  opposed  to  what  happens  in  a non-spectra  I I y opponent 
coll  when  switching  from  dark  to  light.  He  will  say  that  it  appears 
that  simultaneous  brightness  contrast  in  part  and  simultaneous  color 
contrast  depend  upon  interactions  that  occur  past  the  LGN  [12J  . 

Ue  now  turn  to  a precise  description  of  the  model  that  can  be 
built  on  these  experimental  data. 


CHAPTER  2 


A HOMOMORPHIC  MODEL  FOR  HUMAN  COLOR  VISION 
2. 1 Cone  Stage 

As  described  in  chapter  1 there  are  three  cone,  types  involved 
in  human  photopic  vision,  each  of  them  containing  a pigment  with 
different  spectral  absorption  characteristics.  From  color-matching 
experiments,  the  CIE  ("Commission  Internationale  de  I ’ ec I a i rage" ) 
derived  a set  of  three  color  matching  functions  which,  according  to 
Grassman  laws  of  color  mixture,  have  to  be  a linear  combination  of 
the  pigments  absorption  curves. 

Of  course  there  is  an  infinite  number  of  such  comb i na t i ons  but 
we  are  restricted  by  the  fact  that  the  resulting  curves  must  be 
positive  in  the  visible  spectrum  and  have  a single  maximum  at  a 
wavelength  close  to  the  ones  indicated  by  the  methods  referred  to  in 
chapter  1.  Nonetheless,  despite  these  important  restrictions  on  the 
linear  transformat i on  there  are  quite  a few  systems  of  so-called 
"fundamentals"  in  the  I i terature  (20,  30,  613  . 

The  system  we  chose  is  the  one  proposed  by  Stiles  (47,  483. 
The  three  resulting  absorption  curves  I (N) , m (M , s (N)  are  positive  in 
the  visible  spectrum  and  have  single  maxima  at  respectively  57S,  540 
and  445  nm.  They  are  shown  in  figure  2.1.  The  exact  linear 


transformat i on  that  relates  those  spectral  absorption  curves  to  the 
CIE  xC\),  y(*3  and  z(\)  curves  i6  given  in  appendix  C. 


Fig.  2.1-  The  cones  spectral  absorption  functions  1(X),  m(X), 
s(X) . 
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Fig.  2.2-  A view  of  a constant  A plane:  the  axis  runs 

horizontally,  the  C,,  axis  vertically.  The  curved 

boundaries  are  due  to  the  logarithmic  stage  in  the 
model  and  to  the  fact  that  only  a limited  gamut  of 
colors  can  be  reproduced  in  the  primary  system  used. 
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Ue  are  now  in  position  to  give  a precise  description  of  the 
cono  absorption  stage  of  the  model.  Let  I(x,y,X)  be  an  image,  where 
x and  y are  the  spatial  coordinates  and  X is  the  wavelength  of  the 
light.  Then  the  absorption  of  this  light  energy  by  the  three 
pigments  yields  the  following  three  signals: 

L(x,y)»J'  I (x,  y , X)  I (X)  dX  (2.1a) 

I'Ux.yJ-J'  I (x,y,X)m(X)dX  (2.1b) 

S(x,y)=J'  I (x,  y , X)  s (X)  dX  (2.1c) 

where  each  integral  is  taken  over  the  visible  spectrum  (from  380  to 
£80  nm) . 

The  nonlinear  response  of  the  cones  transforms  those  signals  in 
the  following  way: 

L*(x,y)  - Log(L(x,y))  (2.2a) 

HMx.y)  - Log (M (x, y) ) (2.2b) 

S*(x,y)  - Log  (S  (x, y) ) (2.2c) 

It  is  interesting  to  mention  that  the  problem  of  the  exact  type  of 
the  nonlinearity  in  human  cones  has  yet  to  be  solved.  Cornsweet 
CGI  . for  instance,  advocates  a nonlinearity  of  the  type  U*I/(V+I) 
where  U and  V are  constants  and  I is  the  input  intensity.  His 
reasons  are  based  on  a model  of  the  chemical  processes  taking  place 
during  the  cone  absorption  stage  which  predicts  such  a law.  He  also 
points  out  that  over  a large  range  of  intensities  this  nonlinearity 
and  the  logarithm  are  in  close  agreement.  Indeed  it  may  be  noted 
that  the  assumption  of  an  exact  logarithmic  form  for  the 

nonlinearity  i6  not  a critical  feature  as  far  as  psychophysics  goes 
and  that  over  large  ranges  of  the  variables  other  simple  functions 
behave  in  much  the  same  way.  But.  as  we  will  see  in  paragraph  2.7, 


from  an  image  processing  standpoint  the  situation  is  quite  different 
and  explains  why  we  chose  the  logarithm  apart  from  its  computational 
& i mp I i c i t y . 

Before  we  leave  the  topic  of  the  absorption  of  light  in  the 
cone  receptors  and  of  the  nonlinearity  that  immediatly  follows  it, 
we  would  like  to  mention  one  more  fact.  The  light  which  is  absorbed 
at  the  receptor  level  has  already  travelled  through  the  optic  media 
(cornea,  aqueous  humor,  crystalline  lens  and  vitreous  humor)  causing 
a blurred  retinal  image. This  can  be  modelled  as  the  result  of  a 
linear  space- i nvar i ant  filtering  operation  whose  modulation  transfer 
function  has  been  measured  [5)  and  is  character  i zed  by  a high 
frequency  attenuation.  Another  consequence  is  the  introduction  of 
chromatic  aberrations  where  the  image  for  shorter  wavelengths  lies 
slightly  in  front  and  for  longer  wavelengths  slightly  behind  the 
retina.  Ue  will  neglect  this  last  effect  in  what  follows. 


2.2  Separation  of  chromatic  and  achromatic  information 

Ue  showed  in  chapter  1 that  there  is  strong  neurophysiological 
evidence  in  favor  of  the  existence  of  separate  chromatic  and 
achromatic  channels  in  the  human  visual  path,  each  gaining  its 
information  from  the  same  receptors  but  processing  it  in  different 
ways.  The  studies  we  referred  to  suggested  a mathematical 
expression  for  the  achromatic  response,  we  will  call  it  A,  and  the 
two  chromatic  responses,  we  will  call  them  C,  and  C7: 

A-a  (oL"-t-/3!1*+vS*)  -a  (aLog  (L)  +(3Log  (M)  +vLog  (S) ) (2.3a) 

C,«u,  (l"-M*)  -u,Log  (L/M)  (2.3b) 

C,-u,  (L*-S*)  -u,Log  (L/S)  (2.3c) 
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A is  a mathematical  description  of  the  response  of  (+BI-Llh)  and 
(+Wh-BI)  cells  which  add  the  Log-output6  of  the  L-,  fl-  and  S-cones. 
C,  and  C,  are  mathematical  descriptions  of  the  responses  of  (+R-G) , 
(+G-R)  and  (+B-Y) , (+Y-B)  cells  which  subtract  the  Log-outputs  of 
the  L-  and  M-cones  and  L-  and  S-cones  respectively.  The  relation 
between  A,  C,,  C,  and  L",  tt*.  S"  is  thus  a linear  one.  The 
correspond i ng  matrix  1/4  is  given  in  appendix  C.  a,  a,  (Z,  v,  u,,  u7 
are  constants  which  will  be  defined  later. 

2.3  Summation  and  contrast  effects 

Two  important  features  of  the  way  information  is  processed  in 
the  visual  system  have  not  yet  been  included  in  the  model.  The 
first  one,  which  is  responsible  for  the  limited  resolution  of  the 
human  vision,  is  caused  by  the  finite  density  of  receptors  in  the 
retina  and  spatial  summa t ion  of  neura I activity  at  different  stages 
on  the  visual  path  (neural  blur).  A6  we  mentioned  earlier,  another 
cause  of  this  limited  resolution  is  the  optics  of  the  eye  (optical 
blur).  The  neural  blur  effect  can  be  modelled  as  a linear  system 
charac ter i zed  by  an  attenuation  of  high  frequencies  [1,  53. 

The  second  feature  is  the  existence  of  lateral  inhibition 
between  receptors  and  neural  cells  at  different  places  in  the  visual 
path  (retina,  LGN,  cortex).  This  lateral  inhibition  effect  by  which 
a cell  inhibits  the  activity  of  neighboring  cells  with  a strength 
proport ional  to  its  excitation  and  decreasing  with  the  distance  is 
very  much  linear  and,  as  shown  by  many  authors  Cl,  8,  24),  has  a 
sharpening  effect.  It  is  character i zed  by  a low-frequency 
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3-  The  homomorphic  model  of  color  vision  as  suggested 
by  neurophysiological  and  psychophysical  evidence. 


Fig.  2.4-  Stockham's  homomorphic  model  of  achromatic  vision 
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Fig.  2.5-  The  breakdown  of  re f 1 ec ta nee - i 11  urn  i na t i on  separa- 
tion. The  model  of  color  vision,  just  as  the  eye, 
does  not  exactly  separate  illumination  from 
reflectance  since  the  logarithmic  state  operates 
on  integrals  of  products  rather  than  on  products. 
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at  tenuat i on. 

Thus  the  total  effect  of  the  neural  activity  can  be  considered 
3G  the  result  of  the  cascade  of  those  two  systems,  the  neural  blur 
attenuating  high  spatial  frequencies  and  lateral  inhibition 
attenuating  low  spatial  frequencies. 

Although  both  phenomena  seem  to  be  distributed  all  along  the 
visual  path  and  sometime  difficult  to  separate  from  a physiological 
standpoint,  we  will  assume  here  that  we  can  approximate  reality  in  a 
sufficiently  precise  manner  by  modelling  those  effects  by  three 
linear  space  invariant  filters  acting  on  signals  A,  C,  and  C7. 

Let  H(fltf7),  H,  (f4,f7),  H,  ( f » . f 2)  be  the  frequency  responses 
of  those  filters  where  f,  and  f7  are  the  two  spatial  frequencies. 
We  will  furthermore  assume  that  each  of  these  functions  is  the 
product  of  two  functions  correspond i ng  to  lateral  inhibition  and 
neural  blur.  In  other  words  we  dichotomize  the  two  effects  in  every 
channel  by  considering  each  filter  as  the  cascade  of  two  filters 

mode  I I i ng  respectively  lateral  inhibition  and  neural  blur.  The 

correspond i ng  point-spread  functions  are  denoted  by  h(x,y),  h,(x,y) 
and  h7(x,y).  The  perceptual  i nterpretat i on  of  those  filters  will  be 
discussed  more  in  paragraph  S and  in  chapter  3.  They  yield  outputs 
A*.  C,"  and  C7*  given  by: 

A*(x,y)»J'  J*  A (x-p, y-r) h (p,  r)  dpdr  (2.4a) 

C» * (x , y ) » £ C,  (x-p,  y-r)  h,  (p,  t)  dpdr  (2.4b) 

C,*(x,y)  « X J C7(x-p,y-r)h7(p,T)  dPdT  (2.4c) 

Ue  are  now  in  a position  where  we  can  draw  a complete  block 
diagram  of  our  model.  This  is  shown  in  figure  2.3. 
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2.4  Relation  to  other  models 

This  model  can  be  classified  as  an  "opponent-color"  model  in 
the  line  of  the  theory  proposed  first  by  Hering  [25] . He 
hypothesized  three  paired  and  opponent  visual  qualities, 

white-black,  red-green  and  yellow-blue,  independent  of  each  other. 
His  theory  has  had  many  followers  who  tried  to  make  it  quantitative 
in  different  ways.  The  most  recent  efforts  in  this  direction  have 
boen  made  by  Jameson  and  Hurvitch  [2S] , Meessen  [34],  Shk I over  [46]  , 
Koonderink  et.  al.  [31]  and  Frei  [17].  But  thi6  is  to  our 

knowledge  the  first  time  that  such  a formalism  is  used  for  image 
processing  and  for  an  experimental  study  of  color  contrast  effects. 

Further  this  model  is  related  to  a model  for  achromatic  vision 
that  has  been  proposed  and  used  for  image  processing  by  Stockham 
[50].  For  an  achromatic  image  the  outputs  of  the  cone  absorption 
stage  are  by  definition  equal: 

L(x,y)=f1(x,y)«S(x,y) 

and  thus: 

Ct  (x,  y) -C,(x,  y) -0 

In  that  case  only  the  achromatic  channel  A is  active  : 

A-a  (a+|3+Y)L"(x,  y) 

and  figure  2.3  reduces  to  figure  2.4  which  is  exactly  Stockham's 
model.  This  implies  a very  important  fact  namely  that  we  know  the 
frequency  response  H(f,,f,).  It  is  the  amplitude  modulation 
transfer  function  measured  by  Davidson  [8],  Baudelaire  [1]  and 
others. 

2.5  The  problem  of  brightness  perception 
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In  order  to  discuss  the  perception  of  brightness,  we  will  have 
to  go  back  to  the  CIE  observer  and  examine  more  closely  another 
visual  property  he  embodies  and  which  bears  on  the  perception  of 
brightness.  The  property  in  question  is  expressed  by  the  basic 
principle  of  photometry,  the  additive  law  for  brightness  a 1 60  called 
Abney’s  law.  According  to  tnis  law,  the  condition  for  a match  in 
brightness  of  two  stimuli  of  radiances  (spectral  energy 
distributions)  R(A)  and  FT  (A)  takes  the  form: 

S R(A)V(A)dA  - S R’ U)V(A)dA  (2.5) 
the  integrals  being  taken  as  usual  over  the  visible  spectrum. 

V (A)  is  a function  of  wavelength  known  as  the  relative  luminous 
efficiency  function.  There  are  several  methods  for  measuring  this 
function,  all  involving  monochromatic  stimuli,  many  of  them  are 
reviewed  in  Guth  123).  They  yield  similar  although  not  identical 
resu  Its. 

The  standardized  V (A)  curve,  result  of  the  CIE  measurements,  is 
overywhere  positive  in  the  visible  spectrum  and  has  a single  maximum 
normalized  to  1 at  555  nm. 

Let  us  now  investigate  the  meaning  of  equation  (2.5)  in  terms 
of  perception: 

a)  if  two  patches  of  light  P and  P’  of  radiances  R(A)  and  R’  (A) 
match  in  brightness  then  the  two  patches  of  radiances  cR  (A)  and 
cR’ (A)  still  match  in  brightness  (scalability  property). 

b)  if  patch  P0  matches  patch  Pt  and  also  matches  patch  P*  then  the 
additive  mixture  of  patches  P,  and  P,  will  match  patch  P0 
increased  to  double  its  original  radiance  (additivity  property). 

For  an  observer  who  embodies  this  property  and  makes  color-matches 
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according  to  Grassmann’ s laws  of  additive  color  mixture  it  can  be 
shown  that  the  function  V ( X)  must  be  a linear  combination  of  the 


color-matching  functions  and  thus  also  of  the  cones  spectral 
absorption  curves.  This  fact  together  with  the  Knowledge  that  the 
cone  responses  are  nonlinear  has  lead  people  to  assume  that  V (X)  uas 
one  of  the  responses  Cl 73 . Of  course  this  is  in  contradiction  with 
our  current  Knowledge  about  these  fundamentals  which  we  Know  to  be 
peaKing  at  445,  540  and  575-580  nm  while  V (X)  peaKs  at  555  nm. 

Nonetheless  our  model  should  account  for  the  measurements 
yielding  to  the  function  V (X) . One  method  that  can  be  used  to 
measure  the  V (X)  function  is  called  the  step  by  step  method.  Two 
juxtaposed  monochromatic  lights  of  slightly  different  waveiength6 
are  viewed  and  the  radiance  of  one  is  varied  until  the  total 
"difference"  between  the  patches  is  minimum.  The  wavelength 
difference  can  be  made  so  small  that  the  difference  in  color  at  this 
minimum  setting  is  barely  perceptible.  This  procedure  is  then 
repeated  step  by  step  along  the  spectrum. 

A relative  luminous  efficiency  function  can  be  derived  from  our 
model  by  a corresponding  procedure  as  explained  in  appendix  A.  The 
resulting  function  that  may  be  called  V*(x)  is  related  to  the  cone 
spectral  absorption  curves  by  the  following  equation: 

V(X)-cl  (X)0/“4fl‘r  m(X)‘,/“*B‘r  sMT,a'°'T  (2.5) 
where  the  constant  c has  been  adjusted  to  yield  a maximum  of  1.  A 
comparison  between  V*(X)  and  V (X)  is  shown  in  figures  2.5  and  2.7  on 
a linear  and  a logarithmic  scale  for  the  following  values  of  the 
parameters  a,  (3  and  V: 


a-. 512  <3-.  3G3 


Y-.019 
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Fig.  2.6-  Comparison  between  the  C.I.E.  relative  luminous 

efficiency  function  V(X)  and  V*(X)  predicted  by  the 
model:  on  a linear  scale  (a)  and  a Log-scale  (b).  The 
values  of  V*(X)  are  larger  than  the  values  of  V(X) 
in  the  blue  end  of  the  spectrum,  in  qualitative 
agreement  with  results  obtained  by  Judd. 
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(in  particular  a+0+v-l). 

The  reader  will  notice  that  the  values  of  V*  (>>)  are  greater 
than  the  values  of  V (*)  in  the  blue  end  of  the  spectrum.  This  is 
not  to  be  considered  as  a handicap  for  our  model,  on  the  contrary, 
since  there  is  plenty  of  evidence  that  the  CIE  curve  is 

underestimated  in  this  part  of  the  spectrum  (see  Wyszecki  and  Stiles 
[611.  pp. 429,  435). 

Clearly  we  cannot  say  anything  about  Abney’s  law  as  long  as  we 
do  not  give  a precise  definition  of  brightness  in  the  framework  of 
our  model.  We  showed  that  the  visual  information  is  conveyed  from 
the  retina  to  the  brain  via  three  independent  channels  one  of  them 
transmitting  the  achromatic  or  black  and  white  information.  This 
together  with  the  aforementioned  fact  that  for  an  achromatic  image 
only  this  channel  was  excited  makes  it  reasonable  to  say  that  it 
carries  the  brightness  information.  Thus,  for  a given  color  □ 
defined  by  its  tristimulus  values  L,  H,  S,  which  in  turn  uniquely 
define  the  values  of  A,  C,,  C?,  the  brightness  is  proportional  to  A 
and  honce  to  oLog  (L)  +(3Log  (H)  +Y|_og  (S) . 

We  can  see  at  first  that  with  this  definition  brightness  obeys 
the  scalability  condition.  Indeed  if  patch  P of  tristimulus  values 
(L, M, S)  matches  in  brightness  patch  P’  of  tristimulus  values 
(L’.M’.S’)  then: 

oLog  (L) +|3Log  (l“l) -t-YLog  (S)  - aLog  (L’ ) +0Log  (M’ ) +Yl_og  (S’  ) 
and  patches  of  tristimulus  values  (cL.dl.cS)  and  (cl_* , ctl’ , cS' ) still 
match  in  brightness. 

On  the  other  hand  the  add i t i v i ty  proper ty  is  lost  which  means 
that  in  our  model  brightness  perception  does  not  completely  follow 


Abney’s  lau.  Now  careful  studies  of  the  validity  of  the  law  have 
been  conducted  [23]  and  show  that  Abney’s  law  does  not  hold 
especially  for  direct  brightness  matches.  The  authors  further  add 
that  there  does  not  seem  to  be  an  obvious  replacement  for  the  law 
because  some  colors  are  always  subadditive  (red  and  green)  while 
others  are  subadditive  at  threshold  and  superadditive  at  high 
intonsity  levels  (violet  and  green). 

2.S  None  about  the  perceptual  i nterpretat i on  of  the  (A,C,,C?)  space 

As  can  be  seen  in  figure  2.3,  the  model  takes  as  its  input  an 
intensity  image  Hx.y.A)  which  is  then  processed  to  yield  the  three 
perceptual  quantities  A(x,y),  C,(x,y),  C?(x,y)  and  eventually 
A"(x,y),  C,"(x,y)  and  C,*(x,y)  which  we  will  conveniently  describe 
as  two  three-dimensional  vectors  P(x,y)  and  ?*(x,y). 

Let  us  ignore  the  spatial  filtering  which  changes  the  values  of 
the  perceptual  quantities  but  not  their  i nterpretat i on  and  let  us 
also  ignore  for  simplicity  the  spatial  dependency.  Every  color  Q of 
tristimulus  values  (L,M,S),  which  we  will  also  describe  as  a 
three-dimensional  vector  ?,  is  also  defined  by  a "perceptual  vector" 
P - lAXpC,)'. 

We  have  already  discussed  the  interpretation  of  A with  regard 
to  brightness  and  we  will  just  add  to  what  has  already  been  said 
that  a’  value  of  zero  corresponds  to  a neutral  grey  while  increasing 
positive  values  of  A correspond  to  brighter  and  brighter  stimuli  and 
decreasing  negative  values  correspond  to  darker  and  darker  stimuli. 

If  we  now  consider  colors  for  which  the  value  of  A is  constant 
one  may  ask  how  is  the  variation  of  and  C?  related  to  the 
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perception  of  hue  and  saturation.  Ule  uill  say  that  the  perception 
of  saturation  is  related  to  the  distance  bQ  of  the  point  Q to  the 
A-axis  while  the  perception  of  hue  is  related  to  the  angle  © (figure 
2.7) . 

Ulhen  we  maKe  these  assumptions  we  are  defining  a euclidian 
vector  space  structure  on  the  (A.Cj.Cj)  space  which  allows  us  to 
talk  about  adding  vectors  and  about  distances  and  angles.  Ule  will 
show  in  chapter  5 that  the  norm  does  not  have  to  be  euclidian  and 
that  other  choices  are  possible.  Ue  will  come  back  to  the 

i nterpretat i on  of  vector  addition  with  regard  to  the  structure  of 
images  in  paragraph  2.7  and  concentrate  now  on  what  may  be  cal  led 
the  metric. 

Saturation  is  proportional  to  the  distance  Oq  (see  figure  2.7) 
and  is  thus  related  to  the  sum  of  the  activities  in  the  Ct  and  C? 
channe I s while  hue  is  related  to  the  ratio  of  these  activities.  The 
distance  GQ  of  the  point  representing  the  color  to  the  origin  0 of 
the  (A,C,,C^)  space  is  also  of  interest: 

OQ  - (0q2  + Ob')*'2 

It  is  a measure  of  Doth  the  brightness  and  the  saturation  of  the 
color  or  equivalently  of  the  total  activity  in  all  three  channels. 
Following  Stockham  we  might  call  it  the  strength  of  the  color  Q. 

At  constant  brightness,  lines  of  constant  hue  are  straight 
lines  radiating  from  and  perpendicular  to  the  A-axis  and  lines  of 
constant  saturation  are  circles  centered  on  this  axis.  It  is 
interesting  to  represent  those  lines  in  the  CIE  (x,y)  plane  as  in 
figure  2.8.  The  resulting  network  has  the  same  general  shape  as  the 
one  obtained  from  color-order  systems  such  as  tlunsell  or  Din.  To 
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give  an  idea  of  what  we  mean  by  planes  of  constant  A or  constant 
brightness,  such  a plane  is  shown  in  figure  2.2.  Each  point  of 

coordinates  (C4,C^)  in  this  plane  is  represented  by  a perceptual 
vector  f3  - [a.ChC,!’  where  A is  the  same  for  all  points.  This 
perceptual  vector  uniquely  defines  a tristimulus  vector  ? - (l.M.S)' 
which  is  represented  on  a film  by  different  dyes  concentrations. 
The  curved  boundaries  are  due  to  the  limited  gamut  of  colors 
reproducible  by  the  display  primaries. 

Another  very  important  property  of  the  (C,,C,)  plane  is  the 
following:  it  is  a close  approximation  to  a uniform  chromaticity 

piano  which  has  the  characteristic  that  at  a given  point,  the  locus 
of  all  points  correspond i ng  to  a just  noticeable  difference  in  hue 
is  a circle  of  radius  1 centered  on  the  point.  Similarly,  the 

(A,C,,C^)  6pace  is  a uniform  perceptual  space  in  the  sense  that  at  a 
given  a point,  the  locus  of  all  points  correspond  i ng  to  a just 
noticeable  difference  in  perception  (brightness  or  chromaticity)  ie 
a sphere  of  radius  1 centered  at  that  point.  This  is  achieved  as 
explained  in  appendix  B by  adjusting  the  values  of  parameters  a,  u, 
and  u,  of  equations  2.3a,  2.3b  and  2.3c,  respectively. 

2.7  Discussion  of  the  label  homomorphic 

Homomorphic  is  an  adjective  that  is  used  in  mathematics  where 
people  study  special  functions  called  homomorph  i sms.  Grossly 
speaK i ng  they  are  funct  ions  from  one  set  to  another  that  are 
compatible  with  the  laws  of  composition  of  elements  in  both  sets. 
For  example  the  logarithm  function  is  a homomorphism  of  the 
multiplicative  group  of  real  positive  numbers  onto  the  additive 
group  of  all  real  numbers. 
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In  the  engineering  field  homomorphic  systems  that  use  these 
abstract  ideas  have  been  studied  by  Oppenheim  and  others  [351  and 
proven  to  be  very  powerful  tools  by  which  the  basic  theory  of  linear 
systems  has  been  extended. 

In  particular,  Stockham  [50,  511  proposed  a multiplicative 
homomorphic  model  for  processing  achromatic  images  that  matched  both 
the  structure  of  images  and  of  the  human  visual  system.  The 
multiplicative  process  of  image  formation  (product  of  illumination 
i ( x , y ) and  reflectance  r(x,y))  is  harmoniously  undone  by  the  Log 
sensitivity  of  the  retinal  receptors  that  maps  this  product  into  a 


sum  of 

two  components 

that 

can  later 

on  be  separated 

by  1 i near 

f i 1 tori 
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s i nee 

in  genera 

1 their  spatial 

frequency 

content  tends  to  be  distinct. 

This  analysis  applies  to  black  and  white  images  but  partly 
breaks  down  for  color  since  it  ignores  image  wavelength  content. 
Indeed,  the  image  formation  is  a multiplicative  process  that  yields 
the  product  of  the  scene  illumination  i(x,y,M  and  reflectance 
r(x,y,A)  but,  before  the  visual  system  processes  this  product  with  a 
logarithm,  the  light  is  linearly  absorbed  by  the  three  cone 
pigments.  Hence  no  longer  do  we  have  the  logarithm  of  a product  but 
the  logarithm  of  a sum  of  products  which  is  considerably  different. 
This  situation  is  depicted  in  figure  2.5  for  the  L-cones. 

At  first  glance  it  seems  that  the  color  model  is  not  able  to 
account  for  the  automatic  gain  control  properties  of  the  human 
visual  system  as  the  achromatic  model  is.  Nonetheless,  let  us  show 
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uniformly  increase  the  absolute  level  of  illumination  by  letting 
i(x,y,X)  become  ki(x,y,X)  where  k is  a constant  I arger  than  1.  Then 
it  is  clear  that  L(x,y),  M(x,y)  and  S(x,y)  are  changed  to  kL(x,y), 
kH(x,y)  and  kS(x,y)  respectively.  In  other  words  all  three  cone 
outputs  increase  by  the  same  amount  and  thus  the  chromat  i c i ty 
channels  outputs  C,  and  C*  don’t  change  while  the  brightness  channel 
output  A is  biased  by  the  constant  aLog(k)  (equations  2.3a  to  2.3c), 
which  in  turn  is  readily  eliminated  by  lateral  inhibition  (or 
synonymously  spatial  filtering  with  low  frequency  attenuation). 

Now  suppose  we  not  only  change  the  absolute  level  of 
illumination  but  also  the  color  of  the  illuminant  so  that  we  can 
approximately  write  that  L(x,y),  M(x,y»  and  S(x,y)  become 

respectively  IL(x,y),  mfl(x,y)  and  sS(x,y)  where  l,m  and  s are 
different  constants.  In  other  words  the  cone  outputs  are  changed  by 
different  amounts  uniformly  over  the  field. 

The  brightness  channel  output  is  biased  by  the  constant 
a (oLog  ( I ) +/3Log  (m) +vLog  (s) ) , again  readily  eliminated  by  lateral 
inhibition.  But,  in  contrast  to  the  first  case,  the  chromat i c i ty 
channels  outputs  are  biased  respectively  by  u,Log(l/m)  and 
u,,Log(l/s)  which  are  non  zero.  In  order  that  this  bias  be  readily 
eliminated  we  must  assume  that  lateral  inhibition  also  occurs  in 
those  channels.  Chapter  3 wi  I I be  devoted  to  show  that  this  is 
indeed  the  case. 

As  a summary,  when  we  go  from  achromatic  vision  to  color  vision 
we  lose  the  exact  match  between  image  structure  as  a product  of  two 
components  and  model  or  processor  structure.  Our  model,  like  the 


eyo,  does  not  exactly  separate 


ref  I ectance 


from  i I I urn i na t i on. 
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chapters  4 and  5 will  show  that  it  can  be  used  to  perform  useful 
processing  tasks  on  images. 

From  this  standpoint,  let  us  model  the  tristimulus  vector 
Tlx.y)  - Cl_(x,y)  ,i“|(x,y) , S (x,  y)  ] * as  a product  of  two  components 
t(x,y)  - [l(x,y),m(x1y),s(x,y)],  and 

t’ix.y)  - [ I * (x,  y) , m*  (x, y) , s' (x, y)  ] 1 so  that  one  has: 

? (x,  y)  - t (x,  y)  . t ' (x,  y) 
or  oqu i va I en t I y: 

L (x,  y)  - I (x, y)  I ’ (x,  y) 

H(x,y)  - m(x,y)m’(x,y) 

S Ik,  y)  - s (x,  y)  s’  (x,  y) 

It  is  clear  from  figure  2.3  that  if  p(x.y)  and  ?’(x,y)  are  the 
perceptual  vectors  correspond! ng  to  t(x,y)  and  t’(x,y)  respectively 
then  ?(x,y)  - t (x, y) . t ’ (x, y)  corresponds  to  F(x,y)  + ?'(x,y).  Thus, 
we  now  have  the  i nterpretat i on  of  vector  addition  in  the  (A,CltC?) 
space,  it  simply  corresponds  to  multiplication  of  tri  stimulus 
va I ues. 

Lot  us  see  how  this  remark  coupled  with  our  a66umpticn  of 
independant  spatial  filtering  on  the  perceptual  quantities  (A.Cj.C;,) 
ib  of  interest  to  an  image  processor.  Suppose  that  t(x,y)  i6  slowly 
varying  and  t’(x.y)  rapidly  varying,  then  the  same  will  be  true  of 
F ( x , y ) and  P’(x,y)  respect i ve I y.  But  since  they  are  now  additively 
combined,  it  will  be  possible  to  separate  them,  at  least 
approximately,  by  means  of  linear  filtering  thus  opening  the  door  to 
many  applications,  some  of  them  described  in  chapter  4. 
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Ue  now  realize  how  the  notion  of  the  model  as  a 3-dimensional 
homomorphic  system  is  important.  It  accounts  for  the  automatic  gain 
control  properties  of  the  eye  and  introduces  new  meaningful  color 
image  processing  techniques. 


CHAPTER  3 


A STUDY  OF  SPATIAL  EFFECTS  IN  THE  CHROHATICITY  CHANNELS 

3.1  Visual  adaptation  and  contrast  effects 

There  are  two  types  of  visual  adaptation.  One  sort  happens 

with  sudden  changes  in  the  intensity  or  chromaticity  of  the 
illumi nant  and  is  a fairly  slow  process  (several  minutes)  related  to 
the  bleaching  of  visual  pigments  [SI.  The  other  sort  that  shows  the 
ability  of  the  human  visual  system  to  adjust  instantaneously  to  the 
illuminant  is  related  to  neural  interactions  past  the  retina.  This 
is  in  this  last  type  of  adaptation  we  are  interested  here  and  will 
attempt  to  model. 

The  phenomenon  of  brightness  constancy  is  such  a type  of 
adaptation  and  shows  the  tremendous  ability  of  the  visual  system  to 
discard  the  spatial  variations  in  intensity  of  the  illuminant.  It 
has  been  shown  by  several  authors  CS,  50]  that  a Log  type 

sensitivity  at  the  receptor  level  followed  by  a spatial  linear 
filtering  character i zed  by  an  attenuation  of  low  frequencies  would 
account  for  this  ability.  Experimental  studies  conducted  by  several 
authors  II,  8]  have  confirmed  this  hypothesis  by  actually  measuring 
the  modulation  transfer  function  of  the  neural  network. 

It  should  be  noted  that  this  low  frequency  attenuation  not  only 
accounts  for  brightness  constancy  but  also  for  brightness  contrast 
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Fig.  3.1-  Simultaneous  color  contrast  in  the  -channel. 

a)  The  two  small  squares  have  the  same  tri stimulus 
values. 

b)  The  two  vertical  stripes  have  the  same 
tristimulus  values. 
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effects  where 

the  brightness  at 

one  point  i6 

i nf 1 uenced 

by 

the 

brightness  of 

neighboring  points 

and  does  not 

correspond 

to 

what 

might  be  expected  from  the  distribution  of  light  intensities  if 
there  was  no  neural  interaction.  Mach  Bands,  simultaneous 
brightness  contrast  illusions,  Herman  Grids,  are  example  of  such 
offocts  and  the  interested  reader  is  referred  to  Ratliff  [37], 
Baudelaire  [1]  and  Baxter  12]  for  a more  complete  coverage. 

Not  only  is  the  visual  system  able  to  discard  variations  in 
intensity  of  the  illuminant  but  also  variations  in  color.  The  light 
of  an  ordinary  light  bulb  is  very  yellow  and  the  light  outside  on  a 
clear  sunny  day  is  very  blue  but  in  both  cases  we  will  call  a piece 
of  white  paper  white  even  though  the  spectral  content  of  the  light 
it  reflects  to  our  eye  is  very  different  in  both  cases.  This 
phenomenon  is  called  color  constancy. 

Just  as  color  constancy  corresponds  to  brightness  constancy, 
brightness  contrast  has  its  conterpart  with  color  contrast.  Here  it 
is  the  hue  and  saturation  at  ore  point  which  is  influenced  by  hues 
and  saturation  at  neighboring  points  (figures  3.1  and  3.2).  Of 
course  both  effects  (brightness  and  color  contrast)  can  occur 
simultaneously  and  it  might  be  difficult  to  separate  them. 

Land's  retinex  theory  is  an  attempt  to  account  for  brightness 
and  color  constancy  [323. In  this  theory  the  log-outputs  of 
neighboring  receptors  are  pair-wise  subtracted  in  an  independent 
manner  for  every  cone  system.  By  adding  these  data  along  a path 
connecting  two  points  on  the  retina  the  ratio  of  the  correspond i ng 
cone  outputs  can  be  computed.  This  permits  a comparison 
independently  of  a uniform  or  quas I -un i f orm  illumination.  But  the 
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result  does  not  depend  on  comparisons  elsewhere  and  therefore  this 
theory  does  not  account  for  either  brightness  or  color  contrast 
effects.  Another  drawback  of  this  theory  is  its  lack  of  algebraic 
structure  which  makes  it  difficult,  if  not  impossible,  to  use  to 
moke  quantitative  predictions. 

Empirical  formulations,  such  as  the  von  Kries  coefficients, 
have  been  presented  to  account  for  the  perceptual  appearance  of 
colors  and  color  induction  [29,  38,  52]  but  it  seems  that  the 

homomorphic  approach  which  already  models  brightness  effects  fairly 


well  is  a more  powerful  and  general  tool. 

Before  getting  more  into  these  questions,  one  might  ask  what 
are  the  practical  implications  of  gaining  an  understanding  of  how 
colors  interact  and  how  the  eye  adapts  to  the  illuminant. 

One  possible  answer  can  be  found  among  the  many  people 
producing  colored  products.  They  have  to  worry  about  how  their 
i i na I product  is  going  to  look  and,  in  order  to  avoid  bad  surprises 
such  as  colors  that  do  not  look  the  way  they  are  supposed  to  because 
another  color  has  been  placed  beside,  those  people  have  had  to  study 
color  induction  problems  empirically.  A good  quantitative  model  for 
such  effects  would  obviously  be  welcome  (Judd  1940,  [29]). 


Another  answer  can  be 

found 

by  looking 

toward 

the  film 

i ndus 

try. 

Color  f i 1 ms 

wh  i ch 

have  a 

fixed  spectral  response  cannot 

adapt 

to 

d i f ferent  i 1 

1 urn  inants  and 

if  a film 

ba 1 anced 

for,  for 

o 

X 

T5 

le. 

tungsten  lamp 

1 i ght 

is  used 

to  take  an 

outdoor 

scene  the 

result  will  be  an  ,ob ject i onab I e color  cast  on  the  final  print  which 
can  only  be  avoided  by  trial  and  errors  in  the  photo  lab  during  the 
printing  process.  In  that  case  again  a good  understanding  of  the 
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adapting  ability  of  the  human  eye  would  be  welcome  and  a step 
forward  toward  an  automatization  of  such  a process  as  well  as  a 
so  I i d\bas i s for  the  handling  of  digital  color  images. 

Since  a spatial  filtering  on  the  chromaticity  signals  in  our 
model  character i zed  by  an  attenuation  of  low  spatial  frequencies 
would  account  for  color  constancy  and  color  contrast,  an  experiment 
was  designed  to  test  this  hypothesis. 

3.2  The  ideas  involved  in  the  design 

The  validity  of  the  multiplicative  homomorphic  model  of 
achromatic  vision  has  been  questioned  for  patterns  with  sharp  edges. 
Davidson  and  Whiteside  [9]  indicated  that  edge  effects  were 
incompatible  with  the  model  predictions.  Their  results  have  been 
confirmed  by  Baudelaire  who  concluded  that  the  existence  of  a highly 
nonlinear  edge  oriented  mechanism  centrally  located  (that  is  to  say 


the  LGN) 

and  inf 

luencing  the 

whole  visual  field 

was 

most 

1.  The 

neura 1 

interact i on 

network  responsible 

for 

most 

brightness  effects  on  the  other  hand  is  quite  linear  in  the  absence 
of  edges. 

We  will  also  adopt  here  a careful  attitude  toward  edges  and  use 
smooth  patterns  when  measuring  H,(f,,f?)  and  H,(f,,f,).  In  other 
words,  we  will  only  use  parf-cns  containing  a few  low  spatial 
frequencies.  Also  we  assume  for  simplicity  that  the  lateral 
inhibition  postulated  in  the  two  chromaticity  channels  of  our  model 
is  the  same  in  all  directions  which  means  that  the  frequency 
responses  H,  and  H,,  and  the  corresponding  point-spread  functions  h, 
and  h ,,  are  circularly  symmetric.  Consequent  I y , they  are  entirely 
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defined  by  their  cross-section  which  allows  us  to  experiment  only 
with  one-dimensional  patterns. 

These  patterns  were  displayed  on  the  face  of  a high  quality 
color  television  monitor,  part  of  a Comtal  display  unit,  driven 
by  a general  purpose  digital  computer.  The  display  was  carefully 
calibrated  so  that  it  was  easy  to  relate  precisely  the  digital 

intensities  to  the  intensities  of  the  red,  green  and  blue  lights 

emitted  by  the  three  phosphors  on  the  face  of  the  tube.  The 

calibration  procedure  is  explained  in  appendix  D. 

It  S,(x)  (i-1,2,3)  is  the  spatial  modulation  of  intensity  for 

the  i th  phosphor  and  pt(A)  the  spectral  distribution  of  the  light 
emitted  by  this  phosphor,  the  image  generated  on  the  face  of  the 

tube  i s 

I lx,  X)  - Z.3iS,(x)p,(A) 

The  correspond i ng  tri stimulus  values  are 
L (x)  - 2. 8 iS.(x)  X p,  (A)  I ( A)  dA 

and  two  other  similar  equations  for  M(x)  and  S(x).  If  we  denote  by 
L,,  H,  and  S,  (i-1,2,3)  the  tristimulus  values  of  the  light  emitted 
by  the  i th  phosphor  (for  example  L,-X  p, (A)  I (A)  dA  ) then  thi6  can  be 
conveniently  rewritten  as  a vector  equation 
T (x)  - U,  §(x) 

where  s’(x)  - [5,  (x)  , s?  (x)  , s,  (x) ) l,  ?(x)  - (l(x)  , fl  (x)  , S (x) ] ’ and  U, 
is  a three  by  three  matrix  whose  column  vectors  are  the  tristimulus 
values  of  the  phosphors.  U,  is  computed  as  explained  by  the 
following  d i agram: 
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Fig.  3.3-  Computation  of  the  U4  matrix  from  matrices  E4  and  D,. 

E,  is  the  matrix  given  in  appendix  C that  relates  the  cone  spectral 
sensitivities  to  the  CIE  color  matching  functions  x(X),  y(>)  and 

z(A).  0,  has  for  column  vectors  the  chromaticity  coordinates  of  the 

lights  emitted  by  the  phosphors  and  is  thus  charac ter i s t i c of  those 
phosphors  and  given  by  the  builder.  Thus  E,  depends  only  on  the 
model  (which  cone  sensitivities  were  chosen)  and  D,  depends  only  on 
the  phosphors.  The  only  relationship  between  the  two  is  through  the 
wh i te. 

Indeed  D,  and  E,  are  not  completely  defined  as  long  as  a 
reference  white  has  not  been  chosen.  For  the  monitor  it  corresponds 
to  equal  drive  signals  on  the  three  guns  (or  equivalently  to  equal 
digital  intensities)  and  for  the  eye  model  to  equal  tri  stimulus 
values  L,  H,  S.  In  other  words  if  £J  is  the  three-dimensional  vector 
whose  components  are  the  chromaticity  values  (x,y,z)  of  the  chosen 
white  (for  the  television  monitor  it  is  usually  D6503)  then  one  has 
1 - E,  u and  I - o,  w 

oo  that 

r - u.  r 

where  1'  is  the  vector  [1,1,  II'. 


The  system  V transforms  the  three  ditital  imaqes 
S.  , S„  S.  into  the  perceptual  signals  A,  C 1 , C2 

u | «5 


The  system  V ^ transforms  the  three  perceptual 
signals  A,  C ^ , C 2 back  into  the  three  digital 

images  S, , S,  , S,  . 
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Tho  cimple  algebraic  procedure  used  i6  explained  in  appendix  C. 

This  completely  defines  U,  and  thus  the  relationship  between 
the  three  digital  images  g(x)  and  the  tri6timulus  vector  ?(x). 

It  should  be  pointed  out  that  we  have  neglected  in  this 
computation  the  effect  of  the  eye  optics  which  as  we  recall  is 
ociui valent  to  a linear  low  pass  filter.  But  since  we  are 
experimenting  with  smooth  patterns  our  approximation  is  justified 

m . 

Ulhat  wo  want  to  achieve  with  the  images  §(x)  is  to  generate 
perceptual  signals  A(x),  C,(x)  and  C?(x)  in  the  visual  system  of  a 
human  observer  that  allow  us  to  study  the  frequency  responses  H,  and 
H,.  This  situation  is  summarized  in  figure  3.4. 

In  other  words  the  problem  is,  knowing  what  A.  C,  and  C,  have 
to  be,  what  are  S,,  S,  and  S„.  This  question  is  easily  answered  by 
inverting  the  model  of  figure  3.4.  If  we  condense  all  the  steps  or 
subsystems  between  (SltS?,Sa)  and  (A,C,,Ca)  and  represent  them  as  a 
system  V then,  since  it  is  a cascade  of  invertible  subsystems,  V 
itself  is  invertible  to  V-1  as  pictured  in  figure  3.5.  So  if  we  know 
what  A,  C,,  C,  signals  we  want  it  is  easy  to  define  the  required  S,, 
S,,  Sa  signals  by  applying  V-‘  to  them. 

The  (A,C,,C,)  pattern  we  used  are  given  by: 


[A-a 

C,-c,+k  (s  i n (2rtf  x)  +as  i n 

(brtfx) ) 

C?mc7l 

(3.1a) 

(Q 

1 

< 

C,»c,  C,-c,+k  ( s i n (2n f x 

) +as i n (Gnf x) ) ] 

(3. lb) 

whore  a 

> Cj,  c/t 

k,  a are  constants,  f 

i s the 

spat i a 1 

frequency  and 

x the  spatial  coordinate.  The  frequency  content  of  such  patterns  is 
limited  to  the  0-frequency  (DC)  component  for  the  two  channels  that 
are  not  probed  (the  achromatic  channel  is  always  one  of  those)  and 


to  f requenc i es  f and  3f  as  well  for  the  chromatic  channel  that  is 
tested. 

As  we  mentioned  earlier  those  patterns  were  generated  in  a 
general  purpose  digital  computer  that  performed  the  required 
simulation  of  the  inverse  of  V and  were  displayed  on  the  face  of  a 
high  quality  television  monitor  driven  by  the  computer. 

The  examples  of  those  patterns  shown  in  this  thesis  use  as  the 
final  recording  media  a piece  of  photographic  film.  Unfortunately 
there  is  no  good  quantitative  model  which,  given  the  spectral 
distribution  of  the  input  light  and  the  character i st i cs  of  the  film 
and  of  the  printing  process  used,  can  predict  the  resulting 
tri stimulus  values  experienced  by  a human  observer  when  looking  at 
the  final  print  illuminated  by  a given  light.  One  attempt  has  been 
made  in  that  direction  by  Wallis  [57]  but  his  results,  although 
encouraging,  show  that  more  research  effort  needs  to  be  put  in  this 
doma i n. 

Consequently  in  opposition  to  the  case  of  the  television 
monitor  it  is  very  difficult  to  relate  accurately  digital 

intensities  to  tri  stimulus  values  for  the  film  media  and  the 
pictures  in  this  thesis  showing  color  illusions  are  intended  only  to 
give  an  idea  of  the  actual  stimuli  seen  on  the  face  of  the  color 
to  I ov i s i on  mon i tor. 

The  shape  of  the  curve  of  equation 

C (x)  -c+k  (s i n(2nfx)  +cts i n (Gnfx) ) (3.2) 

is  shown  in  figure  3.G  for  ot-1,  a >1  and  a <1.  Setting  a equal  to  1 
in  equation  (3.2)  allows  us  to  test  for  low  spatial  frequency 


attonuation  in 
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(b) 
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Fig.  3.6-  The  shape  of  the  curve  of  equation 
C(x)=c+k(sin(2TTfx)+asin(6irfx)) : 

a)  a=l,  the  leftmost  trough  and  the  rightmost 
peak  are  at  the  same  level. 

b)  a>l , the  leftmost  trough  is  at  a lower  level 
than  the  rightmost  peak. 

c)  a<l , the  leftmost  trough  is  at  a higher  level 
than  the  rightmost  peak. 


Fiq.  3.7-  Test  pattern  corresponding  to  a. sinusoidal  stimulus  in  the 
C-|  -channel  and  constant  stimuli  in  the  A-  and  C2  -channels. 

a)  The  value  of  a in  equation  (3.1a)  is  1 and  a simul- 
taneous color  contrast  illusion  can  be  observed. 

b)  The  value  of  a in  equation  (3.1a)  is  0.5  which  corres- 
ponds approximately  to  the  cancellation  of  the  illusion. 


Fig.  3 


.8-  Test  pattern  corresponding  to  a sinusoidal  stimulus  in  the 
C2  -channel  and  constant  stimuli  in  the  A-  and  -channels. 

a)  The  value  of  a in  equation  (3.1b)  is  1 and  a simul- 
taneous color  contrast  illusion  can  be  observed. 

b)  The  value  of  a in  equation  (3.1b)  is  0.6  which  corres- 
ponds approximately  to  the  cancellation  of  the  illusion. 
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the  channel  of  interest.  Indeed  if  we  assume  linear  spatial 
filtering  on  C(x)  characterized  by  an  amplitude  modulation  transfer 
function  K(f)  the  actual  stimulus  experienced  is 

C*(x)  - K(0)c  + k(K(f)sin(2nfx)  + oK  (3f ) s i n (6nf x) ) 
which  can  be  rewritten  as 

C"(x)  - K(0ic  t-  kK(f)  (sin(2nfx)  + a (K  (3f ) /K  ( f ) ) s i n (Gnf  x))  (3.3) 

so  if  K(3f)  is  greater  than  K(f)  and  a equals  1 we  should  experience 
an  illusion  as  indicated  by  figures  3.S,  3.7  and  3.8. 

This  illusion  can  actually  be  cancelled  by  setting  a to 
K(f)/K(3f)  (equation  (3.3)).  This  is  the  key  to  the  measurement  of 
the  low  frequency  attenuation  in  the  chromaticity  channels. 

For  a fixed  spatial  frequency  f,  patterns  corresponding  to 
values  of  a between  1 and  .1  were  presented  to  a set  of  observers 
and  the  value  of  a which  cancelled  the  color  illusion  experienced 
when  a was  equal  to  1 was  a measure  of  the  ratio  K(f)/K(3f).  The 
oxperience  was  repeated  for  four  spatial  frequencies  (f, -0.142, 
f, -3.234,  fa-8.588,  f *-1.13S  eye  I es/oegree)  on  both  chromaticity 

channels.  The  experiment  is  described  in  more  details  in  appendix 

E. 

It  should  be  emphasized  that  all  this  reasoning  is  based  upon 
tho  assumption  of  linearity  of  the  lateral  inhibition  and  that  this 
experiment  is  also  a test  of  that  property. 

3.3  Results  of  the  experiment 

Ue  found  that  in  the  range  of  frequencies  studied  the  fol  lowing 
ro I at i one  held: 

H,(3f)  - 2.0H,(f)  (3.4a) 


| 

: 
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H?(3f ) - 1.7H,(f)  (3.4b) 

which  should  be  compared  with  Baudelaire’s  result  for  H: 

H(3f)  - 2.0H(f)  (3.4c) 

and  Davidson's  result: 

H (3f ) -2.2H (f ) (3. 4d) 

These  relations  imply  a power- 1 aw  for  H,  H,  and  H,  in  the  frequency 
band  tested  and  thus  a linear  relationship  on  a Log-Log  scale.  Thus 
our  conclusion  is  that  there  is  a low  frequency  attenuation  in  the 
chromaticity  channels  and  the  relative  amplitude  modulation  of 
frequencies  f and  3f  is  about  the  same  in  the  red-green  channel  as 
in  the  achromatic  and  a little  less  in  the  yellow-blue  channel  for 
the  band  of  frequencies  studied.  This  is  in  agreement  with  other 
studies  (21)  which  concluded  that  the  contrast  sensitivity  of  the 
yellow-blue  mechanism  is  less  than  the  contrast  sensitivity  of  the 
red-green  and  brightness  mechanisms.  Of  course  relations  (3.4a)  and 
(3.4b)  do  not  completely  determine  H,  and  H,  and  in  particular  say 
nothing  about  very  low  frequencies  (below  .1  cycle/degree)  and  about 
frequencies  higher  than  about  4 cycles/degree. 

But,  just  as  for  brightness  effects  the  important  band  of 
frequencies  is  from  about  .1  to  B cycles/degree  (1),  for  color 
effects  the  important  band  is  from  about  .1  to  2-4  cycles/degree. 

Nonetheless  we  were  able  to  get  at  least  a rough  idea  of  the  , 
shapes  of  H,  and  H?  at  higher  frequencies  as  follows.  First  of  al  I 
in  the  experiment  on  the  C?-channe  I (yellow-blue)  at  frequency 
ft-1.13B  cycle/degree  four  out  the  five  observers  reported  seeing 
the  illusion  in  reverse  that  is  to  say  reported  seeing  the  stripes 
in  the  middle  of  the  blue  bars  bluer  than  the  stripes  in  the  middle 
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of  the  yellow  bars  for  a set  to  1 while  the  fifth  observer  reported 
seeing  the  illusion  much  weaker  than  for  the  other  three 
f requenc i es.  This  effect  is  called  simultaneous  co I or  similitude 
[123.  In  our  case,  this  means  that  for  the  majority  of  observers 
H,(3ft)  was  less  than  H,(fJ  (see  figure  3.S)  thus  allowing  us  to 
roughly  estimate  the  position  of  the  maximum  at  2 cycles/degree. 
The  same  phenomenon  also  happened  in  the  study  of  the  C, -channel  but 
for  a higher  frequency  (2.272  cycles/degree)  which  also  allowed  us 
to  roughly  position  the  maximum  of  H,  at  about  4 cycles/degree. 


Past  those 

max i ma 

there 

seems 

to 

be  a ro 1 

lof  f 

toward  higher 

frequencies.  Ue 

i ndeed 

know 

that 

the 

neural  network  between  the 

retina  and  the 

v i sua 1 

cortex 

i s 

character i zed 

by 

two  important 

properties:  lateral  inhibition  which  applies  only  over  a limited 
rancjo  and  whose  transfer  function  is  character  i zed  by  a 

low-frequency  attenuation  and  is  flat  for  high  frequencies 
(high-pass  filter)  is  the  first.  Neural  summation  that  occurs  at 
different  levels  and  whose  equivalent  transfer  function  is 
characterized  by  a high  frequency  attenuation  (low-pass  filter)  is 
the  second.  Although  it  is  clear  that  those  two  processes  are 
physiologically  combined  their  effects  on  visual  patterns  are  quite 
different  and  it  seems  justified  to  separate  them  conceptually. 

Ue  mentioned  ear  I i er  that  the  contrast  sensitivity  of  the 
visual  neural  system  had  been  measured  by  contrast  threshold  methods 
by  Campbell  and  Green  [5).  Baudelaire  used  their  data  for 


frequencies  higher  than  8 cycles/degree  as  a good  estimate  of  the 
modulation  transfer  function  of  the  neural  summation  for  the 
brightness  channel.  Ue  will  also  use  them  here  in  the  following 
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Fig.  3.9-  The  frequency  responses  H(f),  H^f)  and  H^f)  of  the 

filters  operating  on  the  A,  C-|  and  C2  signals  (a,  c,  e), 

respectively,  and  (b,  d,  f)  the  corresponding 
point-spread  functions  h(x),  h^(x)  and  h2(x).  The 

frequency  responses  are  plotted  on  a log-log  scale,  the 
point-spread  functions  for  a span  of  one  degree  of  visual 
angle. 
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way. 

Lot  K(f)  be  the  Campbell  and  Green  data  transformed  as  follows. 
K(f)  is  flat  and  equal  to  1 from  0 to  8 eye  I es/degree  and  for  higher 
frequencies  is  the  previously  mentioned  data.  This  is  what  we  will 
t ake , after  Baude I a i re,  as  the  modu I at i on  transf er  function 
equivalent  to  the  neural  summation  for  the  brightness  channel. 

Ue  will  assume  that  K,(f),  the  modulation  transfer  function 
equivalent  to  neurai  summation  in  the  C,-chromat i c i ty  channel,  is  a 
shifted  version  of  K(f)  (Kt  ( f ) »K  ( 2 f ) ) . This  is  in  agreement  with 
our  finding  of  a maximum  at  about  4 cycles/degree  for  H,(f)  and  with 
the  fact  that  visual  acuity  is  poorer  for  pure  colors  than  for 
br i ghtness. 

In  the  same  way  we  will  assume  that  K, ( f ) is  also  a shifted 
version  of  K(f)  (K?  ( f ) =K  (4f ) ) which,  again,  is  in  agreement  with  our 
finding  of  a maximum  at  about  2 cycles/degree  for  H,(f)  and  with  the 
fact  that  visual  acuity  is  poorer  for  yellow-blue  colors  than  for 
red-green  colors  [563  . 

Next  we  will  combine  our  experimental  data  on  the  low-frequency 
part  of  H,  and  H,  with  the  modified  Campbell  and  Green  data.  The 
resulting  frequency  responses  are  shown  in  figure  3.9  as  well  as  the 
corresponding  point  spread  functions. 

3.4  Relation  to  the  color  Mach  Bands  problem 

The  question  of  whether  it  is  possible  to  induce  Mach  Bands 
with  chromatic  gradients  only  (keeping  brightness  constant)  is  still 
con  trovers i a I . Some  people  reported  negative  results  [15,  18,  553 
tending  to  show  that  lateral  inhibition  was  not  present  in  the 
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chroma t i c i ty  channels  thus  leaving  the  well  verified  fact  of 

simultaneous  color  contrast  completely  unexplained.  Some  other 
poop  I o [22,  28]  reported  that  colored  Mach  Bands  were  visible  on 
color  gradients  with  constant  luminance  but  that  those  effects  were 
weaker  than  in  the  achromatic  case. 

Lot  us  see  how  our  findings  might  provide  us  with  an 

explanation  for  this  controversy.  Comparing  equations  (3.1a)  to 
(3.1c)  and  taking  into  account  the  fact  that  the  peak  frequencies 
aro  8,4  and  2 cycles/degree  for  H,  H.  and  H?  respectively  we  can 
approximately  write 

H,(f)  = H(2f) 

H,(f)  - H (4  f ) 

which  state  that  the  two  chromatic  frequency  responses  are  shifted 
versions  of  the  achromatic  frequency  response.  In  the  space  domain 
those  equations  are  equivalent  to 

h,(x)  - l/2h (x/2)  (3.5a) 

hi, (x)  - l/4h (x/4)  (3.5b) 

Qualitatively,  as  can  be  seen  in  figures  3.3b,  3.3a  and  3.3f  , the 
effect  is  to  broaden  the  point-spread  functions  h4  and  h,  by  a 
factor  of  2 and  4 respectively  with  respect  to  h as  well  as  scaling 
them  down  by  the  same  factors. 

Coming  back  to  the  colored  Mach  Bands  problem,  relations  (3.5a) 
and  (3.5b)  suggest  that  they  should  be  broader  than  the  achromatic 
Mach  Bands  and  consequently  less  visible. 


CHAPTER  4 


APPLICATION  OF  THE  MODEL  TO  IMAGE  PROCESSING 

4.1  Scanning  color  images  in  and  out  of  a computer 

In  order  to  test  the  model  on  real  life  color  images  it  was 

necessary  first  to  digitize  such  images  and  second  to  use  a 

✓ 

scanning-out  procedure  where  a piece  of  color  film  was  exposed  and 
processed  in  a fixed  and  controlled  way.  This  was  done  to  minimize 
display  equipment  and  film  non  I i near  i t i es  as  well  as  the  effects  of 
the  printing  process  on  the  results  of  the  experiments  and  make  sure 
that  what  we  were  looking  at  were  actual  consequences  of  the  digital 
processing  and  not  of  film  processing. 

A color  film  (Kodak  Ektacolor  Professional  Film,  Type  S)  was 
exposed  outdoors  yielding  a color  negative  after  development.  Lie 
made  sure  that  the  dynamic  range  of  the  scene  fitted  that  of  the 
film  so  that  the  only  nonlinearity  in  this  process  was  represented 
by  the  gammas  of  the  D-LogE  curves  of  the  three  dyes  present  in  the 
negative  after  development.  They  were  measured  by  exposing  the  same 
day  a calibrated  grey-scale  and  plotting  the  three  curves  showing 
transmission  density  on  the  negative  versus  reflection  densities  on 
the  grey-scale.  The  next  step  in  the  scanning-in  process  was  to 
prepare  three  black  and  white  prints  from  this  color  negative.  They 
were  obtained  by  exposing  panchromatic  paper  (Kodak  Resisto  Rapid 
Pan)  from  the  color  negative  through  three  Wratten  filters  (Red  92, 
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Green  59  and  Blue  47B) . It  turned  out  that  when  doing  so  we  had  to 
uco  the  whole  dynamic  range  of  the  paper  and  that  consequently  we 
wore  not  using  only  the  linear  part  of  its  D-LogE  curve  as  in  the 
case  of  the  negative  but  also  the  parts  usually  referred  to  as  the 
"toe"  and  the  "shoulder"  thus  introducing  another  nonlinearity. 
Those  three  prints  were  then  scanned  independently  on  a reflectance 
scanner  and  the  two  non  I i near i t i es  mentioned  earlier  compensated  for 
by  software. 

The  net  result  of  this  set  of  operations  was  three  digital 
images  correspond i ng  to  the  red,  green  3nd  blue  light  intensities  in 
the  original  scene.  Since  we  compensated  for  all  non  I i near i t i es  in 
the  scanning-in  process,  the  numbers  in  the  computer  were  actually 
proportional  to  the  light  intensities  incident  on  the  film  when  the 
shutter  was  opened. 

The  scanning-out  process  was  done  in  two  ways.  The  first, 
intended  to  provide  a "quick  look"  at  the  results  of  an  experiment 
was  simply  to  display,  the  red,  green  and  blue  images  on  the  face  of 
the  television  monitor  we  used  for  the  psychophysical  experiments 
doGcribed  in  chaptor  3.  The  second,  intended  to  get  a high-quality 
hard-copy,  consisted  in  displaying  the  red,  green  and  blue  images  on 
the  face  of  a very  high  quality  cathode  ray  tube  (CRT)  and  exposing 
sequent  i a I I y a piece  of  color  film  (Kodak  Ver-icolor  S Film)  through 
throe  filters  (Uratten  filters  number  25,  58  and  47B) . The  film  was 
then  taken  to  the  photo  lab  where  it  was  developed  according  to  a 


fixed  procedure  described  in  appendix  D. 

The  results  of  the  total  scanning-in  and  out  procedures  are 


Fig.  4.1-  The  512  by  512  resolution  original  color  image  "BECKY". 


Fig.  4.2-  The  256  by  256  resolution  original  color  image  "CAR-PORT". 
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shown  for  two  real  life  color  images  "BECKY"  and  "CAR-PORT"  in 
figures  4.1  and  4.2. 

' 4.2  Practical  implications 

Let  us  see  now  how  the  model  developed  in  the  first  three 
chapters  can  be  used  to  process  those  images.  The  obvious  problem 
is  that  the  input  to  the  model  (figure  2.3)  is  I(x,y,*),  a function 
of  wavelength  as  well  as  of  spatial  coordinates,  while  the  only  data 
that  are  available  to  us  are  three  images  R(x,y),  G(x,y)  and  B(x,y), 
results  of  the  scanning-in  process,  where  the  wavelength  information 
has  been  lost. 

But  we  notice  that  we  already  solved  this  problem  in  chapter  3. 
Indeed  in  the  case  of  the  television  monitor  we  knew  the 
chromaticity  coordinates  of  the  lights  emitted  by  the  phosphors  and 
thus  the  matrix  transf ormat i on  from  the  tri stimulus  values 
correspond i ng  to  those  pr i mar i es  into  cone  responses.  In  the  case 
of  the  CRT  we  know  the  character  i st  i c of  the  light  emitted  by  its 
phosphors  (CIE  C i I luminant)  and  this  entirely  defines  the 
chromaticity  coordinates  of  the  light  coming  through  the  three 
Uratten  filters  used.  Thus  in  this  case  again  we  can  easily  compute 
the  matrix  U,  such  that 


IL(x.y) ,H(x,y) .Six.y)]*  - U?  IR (x,  y ) , G (x,  y ) , B (x , y ) ] ’ 


Uo  can  now 
C. 

redraw  figure  2.3  as 

figure  4.3. 

U,  is  g i ven 

in  appendix 

S i nee 

we  have 

a way  of 

mapping  co 

lor  images 

defined  by 

T (x,  y)  - [R  (x 

, y) ,G (x, y) 

.Blx.y)]’ 

into  the 

perceptua 1 

space  as 

P (x,  y)  - [A  (x,  y)  ,Ct  (x,  y)  ,C,  (c,  y)  3 * and  that  furthermore  this  mapping 
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ie  invertible,  it  is  tempting  to  process  P(x,y)  because  we  know  that 
by  doing  60  we  will  actually  be  processing  the  quantities  that  are 
perceptually  important  for  a human  observer.  This  idea  is 
summarized  in  figure  4.7.  From  equations  2.3a  to  2.3c  it  is  easy  to 
der i ve 


+ (A 

- A)/a  + 

(/3/u,)  (£,  - C,)  + 

(Y/U,)  (Cj  - C,) 

(4.1a) 

+ (A 

- A)/a  + 

( ip  - l)/ut)  (£,  - 

C,)  + (Y/u,)  (c? 

- C7) 

(4.1b) 

+ (A 

- A)/a  + 

((3/u,)  (c,  - C,)  + 

( (y  - l)/u?)  (£r 

- C7) 

(4.1c) 

4.3  Brightness  processing 

Using  those  ideas  we  first  experimented  with  the  brightness 
information.  For  instance  figure  4.5  shows  the  effect  of  setting 
A(x,y)  to  a constant  across  the  whole  image  which  now  looks  like  a 
cartoon  with  very  little  variation  in  brightness  but  where 
variations  in  color  have  been  preserved.  The  reader  may  argue  for 
example  that  the  sky  or  the  horse  head  look  brighter  than  the  rest 
of  the  scene.  He  is  however  confusing  brightness  with  strength  (see 
figure  2.1)  • From  equations  4.1a  to  4.1c  it  is  easy  to  understand 

what  happens  to  the  tri stimulus  values.  Since  we  have 
A - constant  - c c,  - C,  Ci  - C7 
they  can  be  rewritten  as 
L*  - L*+a' 
n"  - m*+a’ 

S*  - S*+A ' 

whore  A’  - (c-A)/a  and  thus 
L “ L exp (A’ ) 
ft  - M exp (A’ ) 


| 

i 
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S - 5 exp (A’ ) 
or  in  vector  notation 

? - exp(A’)  T (4.2) 

if  we  remember  that  these  quantities  are  functions  of  the  spatial 
coordinates  x and  y,  the  i nterpretat  i on  of  our  processing  is  as 
follows.  At  a point  (x.y)  on  the  image  where  A is  close  to  c,  the 
chosen  constant  brightness.  A’  is  small  and  thus  exp(A’)  is  close  to 
1 and  we  do  not  change  the  tri stimulus  vector,?  very  much.  If  A is 
loss  than  c,  then  A’  is  positive,  exp(A’)  greater  than  1,  and  we 
increase  the  tristimulus  values  at  that  point.  Finally  if  A is 
greater  than  c.  A'  is  negative,  exp(A')  less  than  1,  and  we  decrease 
the  tristimulus  values  at  that  point. 

In  the  same  line  of  ideas  we  can  also  perform  linear, 
space- i nvar i ant  filtering  on  A(x,y)  where  the  frequency  response  H 
of  the  filter  is  circularly  symmetric  and  has  a low  frequency 

attenuation  as  shown  in  figure  4.6.  Equation  (4.2)  can  be  rewritten 

as 

? - (1/exp (-A’ ) ) ? (4.3) 

Noticing  that  -a’  - (a  - 3)/a  and  that,  as  shown  in  figure  4.6, 

A - A is  a low-pass  filtered  version  of  A,  equation  (4.3)  expresses 
nothing  else  than  what  is  known  to  photographers  as  unsharp  masking. 
Of  course  there  is  nothing  that  can  hamper  us  from  choosing  a 

different  frequency  response,  for  example  with  not  only  a low 

frequency  attenuation  but  also  a high  frequency  amplification  thus 
obtaining  a processing  much  more  general  than  unsharp  masking.  This 
processing,  in  the  example  just  described,  can  be  understood  from 
equation  (4.3)  but  also  f<-om  the  standpoint  of  our  concluding 


R(x*)  G(x,y)  | b(x,  y ) | Original  image 

u, 

1 ' ' L(x,y)  S(x,  y ) ^ 


LOG 


L* ( x ,y ) M*(x,y)  S* ( x . y ) 


L*(x,y)  M* ( x ,y ) S* 


L(x.  y)  M(x  ,y ) S(x,y) 


R(x,y)  G(x,y)  b ( x ? y ) Processed  i mag e 


Fig.  4.7-  The  processing  is  based  on  a mapping  of  images 
into  a "perceptual"  space  defined  by  the  model 
where  one  operates  on  perceptually  meaningful 
quantities.  After  processing,  the  image  is  map- 
ped back  into  the  system  of  display  primaries. 


Original  image 


?(x,y)=t(x,y)."t' (x,y) 


4.8-  Homomorphic  interpretation  when  linear  processing 
is  performed  in  the  "perceptual"  space.  Notice  at 
every  step  the  harmony  between  the  processing  and 
the  structure  of  the  processed  quantities. 


Fig  4 9-  Brightness  enhancement  experiment  on  (a)  "BECKY" 
and  (b)  "CAR-PORT". 


4.10-  Chromatic  enhancement  experiment  on  (a)  "BECKY 
and  (b)  "CAR-PORT". 


(cycles/picture)  f 


(cycles/picture)  f 


(cycles/picture)  r 


Cross-sections  of  the  frequency  responses  of  the 
filters  used  to  enhance  the  chromatic  information 
in  (a)  the  C-j-  and  C^-  channels  for  "CAR-PORT", 

in  (b)  the  C^-  channel  for  "BECKY"  and  in  (c)  the 

C0-  channel  for  "BECKY". 


Fig.  4.12-  Brightness  and  chromatic  enhancement  experiment 
on  (a)  "BECKY"  and  (b)  "CAR-PORT". 
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remarks  in  chapter  2. 

Indeed  if  we  model  as  a product  of  a slowly  varying 

component  t-Cl,m,s]t  and  a rapidly  varying  component  t ’ - [ I ’ , m’ , s’ ] *, 
figure  4.7  can  be  redrawn  as  figure  4.8.  ?(x,y),  t(x,y)  and  t’(x,y) 

are  mapped  through  the  model  respectively  onto 
P(x,y)-[A(x,y)  ,C,  (x,  y)  ,C2(x,y)  ] p (x,y)«[b(x,y)  ,c4 (x, y)  , c7(x,  y)  3 * 

and  p ’ (x,  y)  - lb’  (x,  y)  , c,'  (x,  y)  , c/  (x,  y)  ] ’ such  that 

P (x,  y)  ■>  p (x,  y)  + p’  (x,  y) . In  this  particular  case  we  do  not  process 
the  chromatic  signals  and  thus  Cx  - ct,  £7  - c7,  £,’  - c/  and 
£/  - c/ . The  brightness  signal  A(x,y)  is  the  sum  of  a slowly  and 
rapidly  varying  components  b(x,y)  and  b’(x.y)  which  are  processed 
differently  by  our  filter.  The  slowly  varying  component  b,  equal  to 
a (c<  I %0m’+vs*)  and  correspond!  ng  roughly  to  illumination  brightness, 
is  reduced  by  the  low-frequency  attenuation  thus  achieving  dynamic 
range  compression  while  the  rapidly  varying  component  b*  , equal  to 
a (a  I ' *+0m* *+Ys' *)  and  correspond i ng  roughly  to  reflectance 

brightness,  is  enhanced  by  the  high  frequency  emphasis.  This 

processing  is  the  equivalent  for  color  images  to  Stockham’s  work  on 
black  and  white  images.  Notice  that  according  to  our  model  we 

should  not  be  changing  the  chromatic  content  of  the  original  image. 
Results  are  shown  in  figure  4.9.  The  radial  frequency  response  used 
is  shown  in  figure  4.4. 

4.4  Chromatic  processing 

If  we  now  process  chromatic  information  and  leave  brightness 

nformation  untouched,  supposing  again  that  the  processing  is 


•oar,  we  can  rewrite  equations  4.1a  to  4.1c  as 
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L*-  (l‘*+l2*)  /2+(a-  (S‘+52)  /2)  /a-t-  ((3/2)  (S.'-c.2)  + 

(v/2)  (e,*-e,*J  (4.4a) 

A*-  /2+  (a-  (a‘+A2)  /2)  /a+  ( (0-1 ) /2)  (£/-£/)  + 

(v/2)  (£?j-c2‘)  (4.4b) 

§•-  (s'Vs2*')  /2+  (A-  (a‘+a2)  /2)  /a+  (0/2)  (fi,1-^2)  + 

((l-v)/2) (£/-£/)  (4.4c) 

whore  refers  to  the  filtering  in  the  C,-channel  and  refers  to 
the  filtering  in  the  C7-channe  I . The  meaning  of  those  equations  is 
not  immadiatly  obvious  but  if  we  suppose  for  simplicity  that  the 
linear  filtering  on  C,  and  C7  is  the  same,  then  equations  (4.4a)  to 

(4.4c)  rewrite  very  simply  as 


L*  " Ll*  + 

(A 

- A1)  /a 

(4.5a) 

M*  - Pll*  + 

(a 

- S‘)/a 

(4.5b) 

S*  - S“  + 

(A 

- A‘)/a 

(4 . 5c) 

where,  for 

examp  1 e , 

L1* 

represents  the  fi 

1 tered  version 

of 

L*  by  the 

"chromat i c" 

f i 1 ter. 

So 

the  net  effect 

is  in  this  case 

the  same  as 

per  f orm i ng 

the  same 

1 i near  f i 1 ter i ng  on 

L\  M\  S*  as 

on 

C,  and  C2 

and  have  it  followed  by  masking  on  the  resulting  tristimulus  values. 

A complementary  i h'terpretat  i on  of  this  processing  i6  the 
"homomorphic"  one.  Again  we  model  L,  M,  S as  products  of  slowly  and 
rapidly  varying  components  which  makes  the  perceptual  vector  p the 
sum  of  two  vectors  p and  p’ , the  coordinates  of  which  are 
respec t i ve I y slowly  and  rapidly  varying.  Ue  can  interpret  the 
slowly  varying  part  of  C4  and  C?  as  the  illumination  chromaticity 
and  the  rapidly  varying  part  as  the  reflectance  chromat i c i ty.  Then 
a linear  space  invariant  filtering  on  and  C7  character i zed  by  a 
low  frequency  attenuation  and  a high  frequency  emphasis  will  enhance 
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saturation  of  objects  which  tend  to  be  small  and  discard  what  may  be 
called  the  chromatic  component  of  the  illumination  thus  realizing  an 
automatic  color  balance  control.  This  can  particularly  well  be  seen 
when  comparing  figure  4.2  with  figure  4.10b  where  the  slightly 
yellowish  low-frequency  chromatic  component  has  been  completely 
removed  by  the  filtering  performed  on  the  and  C?  signals. 
Examp  I os  of  6uch  a process  are  shown  in  figure  4.10.  The  radial 

frequency  responses  used  are  shown  in  figure  4.11. 

4.S  Brightness  and  chromatic  processing 

Finally  one  might  want  to  combine  brightness  and  chromatic 
enhancement  as  shown  in  figure  4.12.  If  we  denote  by  A the 
processed  brightness,  equations  4.5a  to  4.5c  can  be  rewritten  as 
L*  - L‘*  + (A  - AM /a 
n*  - n‘*  + (A  - AM/a 
s*  - s"  + (A  - AM/a 

again  we  assume  for  simplicity  that  the  linear  filtering  on  C,  and 
C,  is  the  same.  Note  that  the  masking  is  different  from  the  one  in 
the  case  of  chromatic  processing  only.  The  "homomorphic" 

i nterpre  ta  t i on  is  the  same  as  before  but  now  a combination  of 
paragraphs  3 and  4. 


CHAPTER  5 


APPLICATION  OF  THE  MODEL  TO  IMAGE  TRANSMISSION  AND  CODING 

5.1  The  use  of  the  model  for  defining  a distortion  measure  between 
co I or  images 

Shannon’s  rate-di stort  ion  theory  119,  451  was  originally 

developed  to  handle  such  problems  as  efficient  encoding  of  images 
and  speech.  Since  we  are  interested  here  in  images,  let  us  review 
very  briefly  the  essence  of  his  results  in  that  case. 

Lot  I(x,y,X)  be  an  original  image  that  we  want  to  transmit  over 
some  noisy  channel  and  I(x,y,X)  the  received  image.  Let  also  d(  , ) 
be  a positive  real  valued  function  of  two  images  and  consider  d ( I . T ) 
to  be  the  distortion  that  occurs  when  I is  transmitted  and  I 
received.  The  performance  of  the  system  is  then  measured  by 
d*  - ECd(l.I)] 

where  the  expected  va I ue  E is  taken  over  the  ensemble  of  images  of 
interest.  Shannon’s  rate-d i stor t i on  function  R(d*)  is  a lower  bound 
on  the  transmission  rate  required  to  achieve  average  distortion  d*. 
Moreover,  Shannon’s  coding  theorem  also  states  that  one  can  design  a 
code  with  rate  only  negligibly  greater  than  R (d*)  which  achieves 
average  distortion  d*.  Thus  this  function  R(d*)  exactly  specifies 
the  minimum  achievable  transmission  rate  R required  to  transmit  an 
image  with  average  distortion  level  d*  and  provides  an  absolute 
yardstick  against  which  to  compare  the  performance  of  any  practical 
system  (see  for  example  1191). 
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There  are  three  reasons  why  this  potential  value  has  not  been 
realized  to  date.  The  first  one  is  that  there  does  not  currently 
exist  any  tractable  mathematical  models  for  an  image  source,  the 
Gaussian  one  being  obviously  a poor  choice.  The  second  one  is  the 
difficulty  of  computing  the  rate-d i stor t i on  function  for  other  than 
Gaussian  sources  and  square-error  distortion  measures.  The  third 
one  is  that  a distortion  measure  d(  , ) in  agreement  with  subjective 
evaluation  of  image  quality  is  not  known.  By  this  we  mean  that  if 
I(x,y,.\)  is  a reference  original  image  and  Tj(xty,X),  j - 1,2,..., 
ie  any  set  of  reproduced  images,  then  d(l,TJ)  should  rank  the 
reproduced  images  in  the  same  order  as  the  end  user  of  the  images. 

It  has  been  argued  (Sakrison  and  Mannos  1431)  that  this  might 
be  the  prime  reason  of  the  three  why  rate-di stort i on  theory  is  not 
currently  applicable.  Some  work  toward  the  definition  of  such  a 
distortion  measure  for  black  and  white  images  has  been  carried  out 
after  the  pioneering  work  of  Stockham  [53,  513  by  flannos  [33],  Rom 
[39]  who  used  the  mono-frequency  channel  model  of  human  vision. 
This  work  is  currently  being  extended  toward  including  the  existence 
of  several  frequency  channels  12,  44],  On  the  side  of  color  images, 
very  little  has  been  done  and  part  of  this  work  is  an  attempt  to 
define  such  a distance  between  color  images. 

Ue  will  assume  for  simplicity  that  the  original  image  is 
represented  in  some  set  of  primaries  by  its  correspond i ng 
tristimulus  values  R(x,y),  G(x,y)  and  B(x,y).  Our  reference 
original  will  thus  be 

T(x,y)  - [R  (x, y) , G (x, y) , B (x, y) ] * 


while  the  distorted  version  will  be 


?(x,y)  - [ft  (x,  y)  ,6  (x,  y)  ,B  (x,  y)  ] 
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Wo  know  that 

dlT.T)  - J*  S ( [?(x,y)-?(x,y)]  . [f  (x,  y)  (x,  y)  ] ) *'*  dxdy  (5.1) 

.whore  . stands  for  inner-product,  is  in  very  poor  agreement  with 
subjective  evaluation.  But  we  also  know  a great  deal  about  about 
how  the  visual  system  processes  the  information  it  receives  from  the 
outside  world  and  thus  what  is  important  for  the  human  observer.  In 
other  words  it  seems  very  likely  that 

dCf.?)  - S S ([ft*(x.y)-ft*(x,y)).  [ft*(x,y)-ft*(x,y)]  )*'’  dxdy  (5.2) 
is  a much  better  measure  of  the  distortion  between  f and  T than 
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equation  (5.1).  In  equation  (5.2)  ft*  and  ft*  are  the  perceptual 
vectors  correspond! ng  respectively  to  ? and  T.  This  equation  is 
nice  because  it  measures  distances  in  the  (A*,C,*,C?*)  space  which 
is  the  (A,C,,C,)  space  after  spatial  filtering  by  the  linear  filters 
of  frequency  responses  H,  H,,  H,  (see  chapter  3).  This  means  that 
wo  aro  taking  into  account  both  the  fact  that  the  (A,CJtC*)  space  is 
uniform  in  terms  of  perception  (see  chapter  2)  and  that  different 
sensitivities  to  spatial  frequencies  exist  in  the  three  channels. 

5.2  Agreement  between  distance  and  perceptual  ranking 

Wo  artificially  distorted  an  original  image  ? in  ways  to  be 
described.  This  resulted  in  a set  of  five  disturbed  versions  Tj, 

■4  *♦ 

j - 1,2,. ...5,  of  ?.  For  each  T,  we  computed  d(f,Tv)  - dj  according 
to  equation  (5.2).  Ue  found  that  the  way  most  observers  ranked  the 
images  in  terms  of  similarity  to  the  original  was  the  same  as  the 
one  obtained  by  ordering  the  numbers  d ■ (the  smallest  number 
correspond'! ng  to  the  picture  judged  closest  to  the  original). 


Fig. 


5.1 


- Distorted  versions  of  (a)  "BECKY"  and 
Noise  has  been  added  to  the  components 
to  the  red,  green,  blue  primaries. 


(b)  "CAR-PORT", 
corresponding 


FT 


Fig.  5.2-  Distorted  versions  of  (a)  "BECKY"  and  (b)  "CAR-PORT". 

Noise  has  been  added  to  the  components  corresponding 
to  the  Y,  I,  Q N.T.S.C.  transmission  primaries. 
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Fiq  5.3-  Distorted  versions  of  (a)  "BECKY"  and  (b;  CAR-PORT  . 

Noise  has  been  added  to  the  L*>  M*,  S*  cones  loq-outputs 
in  the  model  of  figure  4.3. 
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Fiq.  5.4-  Distorted  versions  of  (a)  "BECKY"  and  (b)  "CAR-PORT  . 

Noise  has  been  added  to  the  A,  C-j , C2  components  in  the 
model  of  figure  4.3. 
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Fig. 


5.5- 


Distorted  versions  of  (a)  "BECKY"  and  (b)  "CAR-PORT  . 
Noise  has  been  added  to  the  A',  C-|  ’ , C2'  components, 

filtered  versions  of  the  A,  C-| , C2  components  in  the 


model  of  figure  4.3. 
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The  distorted  versions  of  "CAR-PORT"  and  "BECKY"  numbered  from 
1 to  5 are  shown  in  figures  5.1  to  5.5.  The  computed  distortions 
dj,  (j-l,..,5),  normalized  to  an  average  distortion  per  point,  are 
ehown  in  table  5.1  for  "BECKY",  in  table  5.2  for  "CAR-PORT".  They 
correspond  to  the  entry  D,.  Ue  can  see  that  in  both  cases  the 
orclering  i s 5,  4,  3,  1,  2.  For  "CAR-PORT",  for  example,  the  most 
disturbed  image  Y?  is  at  an  average  distance  of  20  perceptual  units 
from  the  original  (remember  that  a distance  of  1 corresponds  to  a 

just  noticeable  difference  in  the  (A,C,,C?)  space).  is  5% 

— * — * — ► 

closer,  Ya  15%  closer,  Tt  30%  closer  and  Ts  35%  closer. 

This  can  be  considered  as  a good  confirmation  of  our  claim  that 
the  distance  defined  by  the  model  is  "good"  in  the  sense  of 
paragraph  5.1.  It  is  interesting  here  to  make  a remark  on  the 
nature  of  the  distance  or  metric  we  defined  on  the  (A,C,,C?)  space. 
l*io  chose  it  to  be  a square-error  or  euclidian  distance  because,  as 
wo  mentioned  in  paragraph  5.1,  it  is  the  one  which  mathematically  is 
tho  moro  tractable.  But  we  must  stress  the  fact  that  from  the 
physiologcal  standpoint  it  is  a little  bit  difficult  to  defend 
because  it  assumes  some  kind  of  mechanism  able  to  sense  the  outputs 


of  the 

three 

channe 1 s 

(which  is  not 

unrea 1 i Stic)  and 

somehow 

to 

compute 

the 

square-root  of  the  sum 

of  their  squares 

(wh i ch 

i s 

unrca 1 i s 

tic)  . 

Another 

distance  which 

wou 1 d be  more  i n 

agreemen  t 

with  wha t we 

know  about 

neural  networks 

can  be  i n forma  1 1 y 

def i ned 

as 

"tho  channel  that  shouts  the  loudest  determines  the  r'espon9e"  and 
formally  as  the  "maximum"  distance  whereby  if  0 is  the  origin  of  the 
perceptual  space  and  Q the  point  of  coordinates  (A,C,,C,)  then 


OQ  - Hax(  IAI , IC.I , IC,I ) 
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whore  I I stands  for  magnitude  of.  Another  distance,  also  more 
dofendable  from  the  physiological  standpoint,  and  which  alike  the 
ouclidian  one  yields  the  interpretation  of  OQ  as  a measure  of  the 
total  activity  in  all  three  channels  would  be 
OQ  - I A I + 1C, I + IC?I 

For  those  two  distances,  the  unit  spheres  locus  of  the  points 
representing  colors  which  are  just  noticeably  different  from  a given 
color  would  be  unit  cubes  and  the  flac-Adam  ellipses  would  map  onto 
un i t-squares.  It  is  pleasant  to  notice,  as  shown  in  tables  5.1  and 
5.2,  that  if  those  distances  are  used  for  computing  the  distortions 
d,  instead  of  the  euclidian  one  the  resulting  ranking  is  the  same 
(the  "maximum"  distance  corresponds  to  the  entry  07,  the  "sum  of  the 
magnitudes"  distance  corresponds  to  the  entry  Da) . 

Let  us  mention  for  completeness  the  fact  that  these  three  norms 
are  special  cases  of  a broad  class  of  vector  norms,  the  Holder-norms 
whore 

OQ  - ( I A I “ + 10,1“  + IC7I“)W“ 

The  sum  of  the  magnitude  norm  correspond  i ng  to  a - 1,  the  euclidian 
norm  corresponding  to  a » 2 and  the  maximum  norm  correspond i ng  to 

Ci  m C<0  , 

Obviously  more  work  needs  to  be  done  in  order  to  test  further 
the  metric  of  the  (A,C,,C7)  space  but  those  results  are  extremely 
oncourag i ng. 

5.3  Color  image  transmission  and  coding 

The  description  of  the  way  images  represented  in  figures  5.1  to 
5.5  have  been  distorted  will  provide  us  with  a transition  to 


Fig.  5.6-  Cross-section  of  the  frequency  responses  of  the 
filters  used  to  obtain  (a)  A'  from  A,  (b) 

from  C-|  , (c)  C2 1 from  C2<  In  each  case  the  low- 

frequency  part  is  the  same  as  for  the  curves 
shown  in  figures  3.9a,  3.9c,  3.9e,  respectively. 
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pre-processor  channel  post-processor 
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"perceptual 

space" 


Fig.  5.7-  Rom's  result  for  transmission  of  an  achromatic 
image  over  a noisy  channel  when  the  error 
criterion  is  a square  error  in  the  "perceptual" 
s 
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post-processor 


Fg.  5.8-  Generalization  of  Rom's  result  to  color  image 
transmission. 
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the  problem  of  image  coding.  Noise  was  added  to  the  three 
components  of  the  images  in  different  domains.  The  domains  were  the 
red,  green,  blue  display  primaries  for  figure  5.1,  the  Y,  1 , Q NTSC 
transmission  primaries  (not  gamma  corrected)  for  figure  5.2,  the  L", 
fl*.  S'*  cone  Log-outputs  for  figure  5.3,  the  A,  C,,  C?  components  of 
figure  3.2  for  figure  5.4  and  the  A’,  C/,  C/  components  for  figure 
5.5.  A’,  C,’,  C / were  obtained  by  filtering  A,  C,,  C,  with  linear 
filters  of  frequency  responses  equal  to  H,  H,  and  H,  of  chapter  3 in 
the  low  frequency  part  up  until  the  maximum  and  then  flat  at  higher 
frequencies  (high-pass  filters)  as  shown  in  figure  5.5.  The  reasons 
for  this  choice  will  be  explained  later.  After  the  noise  was  added 
the  image  was  mapped  back,  if  necessary,  into  the  display  primaries 
system  and  displayed. 

In  all  cases  the  noise  added  to  the  three  components  was 
uncorrelated  from  one  component  to  the  other  and  for  each  component 
it  was  zero-average  uniformly  distributed  white  noise.  Furthermore 
the  peak  to  peak  signal  to  noise  ratio  was  4:1  in  the  case  of 
CAR-POnT  and  2:1  in  the  case  of  BECKY. 

If  we  recall  that  uniformly  quantizing  a signal  with  n bits  is 
approximately  equivalent  to  adding  uniformly  distributed  white  noise 
uncorrelated  to  the  signal  with  peak  to  peak  signal  to  noise  ratio 
2",  it  is  obvious  by  looking  at  figures  5.1  to  5.5  that  with  the 
same  number  of  bits  or  equivalently  the  same  relative  noise  level 
the  quality  achieved  is  much  better  if  we  use  the  perceptual  signals 
A.  C„  C,  or  A*.  C,\  C,’. 

More  formally,  it  has  been  shown  recently  C39I  that  in  the  case 
of  transmission  of  black  and  white  images  over  a noisy  channel,  if 
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one  wants  to  minimize  the  mean  square  error  in'  the  perceptual  space, 
tho  optimum  pre-  and  post-processors  are  as  shown  in  figure  5.7.  F 
and  G are  the  frequency  responses  of  two  spatially  invariant  linear 


f i 1 tors 

and 

H 

is  the 

br i ghtnes6-channe 1 

frequency  response  of 

chapter 

3. 

F 

depends 

on  H,  I (x, y)  and 

the  noise  N,  G is  the 

corrospond i ng  W i ener-f  i I ter.  As  pointed  out  by  Rom,  the  dependency 
on  H is  weak  and  in  all  practical  cases  F is  a high-pass  filter  and 
G a low-pass  filter. 

This  can  be  immediatly  generalized  to  the  case  of  color  images 
ae  shown  in  figure  5.8.  So,  according  to  this  theoretical  argument, 
tho  example  where  we  disturbed  A’,  C,’  and  C/  corresponds  almost  to 
the  optimum  since  the  filters  of  figure  5.G  are  high-pas9  filters 
and  thu6  their  inverses  are  low-pass  filters.  The  results  should 
indeed  look  better  to  a human  observer  than  results  obtained  by 
d i f forcnt  methods. 

Those  arguments  6trongly  support  the  validity  of  our  model  to 
define  a distortion  measure  between  color  images  and  the  more  subtle 
claim  that  when  processing  images  we  should  process  them  after  a 
mapping  so  that  the  processed  quantities  are  as  close  as  possible  to 
the  perceptually  important  quantities  to  a human  observer. 

As  a final  example  we  would  like  to  show  applications  of  this 
model  to  image  coding.  The  idea  was  to  code  the  A,  C,,  C7 
perceptual  components  of  an  image  independently  and  use  the  fact 
that  they  are  spatially  filtered  in  different  ways  by  the  visual 
system.  We  noticed  in  chapter  3 that  the  visual  acuity  for  the 
red-green  channel  seemed  to  be  about  twice  poorer  than  the  visual 
acuity  for  brightness  and  twice  better  than  the  visual  acuity  for 
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TABLE  5.1 

PERCEPTUAL  DISTANCES  FOR  "BECKY" 
• Image  number 


1 

2 

3 

4 

5 

0. 

19.51 

23.04 

16.18 

13.21 

12.59 

0, 

17.53 

21.15 

14.18 

11.04 

10.35 

Da 

26.77 

30.59 

22.90 

19.53 

18.87 

TABLE  5.2 

PERCEPTUAL  DISTANCES  FOR  "CAR-PORT" 
Image  number 


1 

2 

3 

4 

5 

o. 

18.99 

19.67 

16.28 

13.15 

12.57 

D, 

16.79 

17.56 

14.10 

10.77 

10.08 

Da 

26.41 

27.00 

23.47 

19.85 

19.22 

Fig.  5.9-  Coded  versions  of  (a)  "BECKY"  and  (b)  "CAR-PORT" 

corresponding  to  an  average  bit  rate  of  1 bit/pixel. 


31 


the  yellow-blue  channel.  These  properties  were  taken  into  account 
in  the  encoding  process  where  we  used  a spatial  transform  encoding 
mothod  developed  by  Tescher  [53].  Hagnitude  and  phase  of  the 
Fourier  transforms  of  filtered  versions  of  A,  C4,  C?  were  encoded  in 
such  a way  that  the  average  bit-rate  was  3.7  bit/pixel  for  the 
brightness  information,  0.2  bit/pixel  for  the  red-green  information 
and  0.1  bit/pixel  for  the  yellow-blue  information  thus  yielding  a 
total  average  bit  rate  of  1 bit/pixel.  The  image  was  then 
reconstructed  through  the  inverse  of  the  model  and  displayed.  The 
results  are  shown  in  figure  5.9.  Since  the  originals  were  scanned 
in  with  27  bits/pixel,  that  is  9 bits/pixel  for  the  red,  green  and 
blue  information,  the  reduction  factor  is  27:1. 


CHAPTER  S 


CONCLUSIONS 

£.  1 Psychophysics  and  modelling 

Using  the  homomorphic  modelling  approach,  we  have  been  able  to 
show  experimentally  that  spatial  linear  inhibition  is  present  in  the 
chroma t i c i ty  channels  of  the  human  visual  system.  Of  course,  our 
results  hold  only  if  the  model  we  hypothesized  is  correct  since  the 
design  of  the  entire  experiment  described  in  chapter  3 is  based  upon 
the  inversion  of  that  model. 

In  particular,  one  may  argue  that  what  we  called  brightness 
does  not  correspond  to  perceptual  brightness  and  that,  consequently, 
tho  simultaneous  contrast  effects  we  measured  were  partly  or  solely 
duo  to  brightness  contrast.  But  this  criticism  supposes  one  thing, 
that  is  that  there  exists  a good  quantitative  definition  of 
brightness.  Ue  know  this  is  true  for  monochromatic  lights.  It  is 
defined  by  the  CIE  V(N)  function  with  the  necessary  corrections  in 
tho  blue  end  of  the  spectrum  . Ue  were  not  operating  with  that  kind 
of  stimuli  since  the  lights  emitted  by  the  phosphors  are  far  from 
being  monochromatic.  An  alternate  possibility  would  have  been  to 
Keep  the  luminance  constant  where  the  luminance  V of  a stimulus  of 
radiance  P(A)  is  defined  as: 

Y - X RU)V(X)  dN 

But  this  in  turn  assumes  that  Abney’s  law  is  va I i cl  which  we  know  to 
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bo  wrong  [231,  especially  for  direct  brightness  matches  which  are 
the  experimental  conditions  we  were  operating  under. 

Thus,  our  definition  of  brightness  is  not  worse  than  any  other 
definition.  Fur thermore,  it  is  based  on  solid  neurophysiological 
evidence  and  we  were  able  to  show  that  it  predicted  a relative 
luminous  efficiency  function  V“(X)  in  extremely  close  agreement  with 
other  experimental  data.  To  put  it  another  way,  our  definition  of 
brightness  is  right  in  the  only  case  where  there  is  agreement  among 
1 1 io  scientific  community  on  its  definition,  namely  in  the  case  of 
monochromat ic  lights.  As  a summary,  we  will  say  that  the  results  of 
our  psychophysical  experiment  are  in  full  agreement  with  our  model: 
tho  constant  (model)  brightness  patterns  we  used  were  indeed  judged 
as  so  by  tho  subjects.  Those  results  are  also  compatible  with  our 
hypothesis  of  linear  lateral  inhibition  and  summation  in  the 
chromatid ty  channels  which,  we  would  like  to  6tre66  it  again,  was 
fundamental  in  the  design  of  the  experiment. 

Wo  will  not  claim  to  have  solved  all  the  problems  of  color 
vision  but  we  hope  to  have  contributed  to  a better  understanding  of 
the  questions  of  chromatic  adaptation  and  simultaneous  color 
contrast  effects.  At  any  rate,  this  field  is  a most  difficult  one 
to  work  in  because,  to  quote  Rushton,  "whereas  nearly  all  the 
phenomena  of  nature  are  simply  observed,  those  of  sensory  pnysiology 
can  only  be  experienced.  So  in  colour  vision  we  perceive  the 
essential  hollowness  of  formal  scientific  explanation."  And  "the 
content  of  our  hollow  scientific  structure  is  such  stuff  a6  dreams 
dro  made  on:  there  is  nothing  either  green  or  grey  but  thinking 


makes  it  so. 
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G.2  Image  processing 

After  the  model  was  calibrated  using  our  experimental  results 
as  well  as  those  of  others,  it  suggested  several  interesting  ideas. 
First  we  showed  how  the  perceptual  space  offered  a meaningful  way  of 
thinking  about  important  perceptual  parameters  such  as  brightness, 
hue,  saturation  and  a concise  formalism  to  describe  them 
quantitatively.  Second  we  showed  that  the  idea  of  structuring  the 
perceptual  space  as  a vector  space  where  vector  addition  corresponds 
to  tristimulus  values  multiplication,  coupled  with  the  existence  of 
spatial  filtering  in  that  space  accounted  for  contrast  and  constancy 
effects  and  suggested  interesting  image  processing  methods  which  we 
showed  to  be  indeed  succesful  both  for  enhancement  and  transmission 
, and  coding.  Third  we  showed  that  the  introduction  of  a norm  on  that 

space,  which  allowed  us  to  measure  distances  herein,  was  the  key  to 
defining  a distortion  measure  in  agreement  with  perceptual 
ova  I uat i on. 

C.3  Further  research 

From  the  psychophys i o I ogi ca I standpoint,  one  may  try  to  verify 
the  linearity  of  neural  interactions  for  more  complicated  patterns 
than  the  one  which  were  used  in  this  study.  True  two-dimensional 
patterns  could  bo  used  on  C,  and  C7  (we  used  only  one-dimensional 
one),  patterns  exciting  both  C,  and  C,  and  patterns  exciting  all 
three  channels  could  also  be  used. 

It  is  also  likely  that  more  than  three  quantities  govern  our 
perception  of  colors  [15]  in  complex  scenes.  We  defined  a new  one, 
which  we  called  strength,  that  was  related  to  the  total  activity  in 
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all  throe  channels  taut  did  not  investigate  its  role  in  perception 
any  further.  A more  systematic  study  of  such  parameters  may  open 
new  avenues  in  the  fields  of  psychophysics  and  image  processing. 

But  we  think  that  the  real  improvement  of  this  model  and 
similar  ones  will  come  from  a study  of  visual  perception  near  edges. 
Indeed,  Baudelaire  Cl]  showed  that  the  predictions  of  the 
homomorphic  model  for  achromatic  vision  broke  down  for  patterns  with 
sharp  edges.  Of  course  this  is  also  true  of  this  model  which  is 
based  on  the  same  ideas  of  modelling  vision  as  a cascade  of  a 
nonlinear  memory  I ess  stage  (Log)  and  a linear  system.  The  trouble 
is  that  edges  determine  our  perception  of  contours  and  thus  of 
shapes.  That  is  why  we  think  that  if  the  next  step  toward  a better 
understanding  of  the  information  processing  capability  of  the  human 
visual  system  is  a step  in  the  direction  of  trying  to  describe  how 
we  perceive  and  recognize  shapes  and  objects,  the  problem  of  edge 
perception  and  the  fundamental  nonlinearity  it  introduces  in  the 
vision  process  will  have  to  be  solved. 

We  are  still  far  from  being  able  to  explain  how  we  structure 
and  understand  what  we  see  with  what  we  know  about  the  visual  system 
but  it  is  hoped  that  studies  like  ours  are  in  the  right  direction. 
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DERIVATION  OF  THE  RELATIVE  LUMINOUS  EFFICIENCY  FUNCTION 


To 

determine  experimentally 

the 

relative  luminous  efficiency 

f unct i on 

one 

can  use  the  so-call 

ed  step  by 

step  method  where  two 

patches 

of 

monochromatic  lights 

are 

v i ewed 

side  by  side.  The 

wave  1 ong  ths 

of  the  two  patches 

are 

si i ght 

ly  different  and  the 

rad i ance 

of 

one  i s var  i ed  unt  i 1 

the 

tota  1 

perceptua 1 d i f f erence 

between  the  two  patches  reaches  a minimum.  This  procedure  is  then 
repeated  step  by  step  along  the  spectrum. 

Let  us  now  see  how  such  a function  can  be  derived  from  the 
model.  Let  P an  P’  be  two  patches  of  light  of  tri stimulus  values 
(L.M.S)  and  (L’.M’.S*)  and  chromatic! ty  coordinates  (l,m,s)  and 

( I * . iii'  , s' ) where 

i - L/(L+n+s>  - l/i  i'  - L'/iL'+rr+SM  - l'/z* 

and  four  other  similar  expressions  for  m,  m’ , s,  s’. 

Lie  are  going  to  make  I’  vary  while  keeping  I’,  m* , s’  fixed 

(change  the  radiance  of  patch  P'  without  changing  its  color). 

Patches  P and  P'  are  represented  in  the  (A,C,,C,)  space  by  two 

points  Q and  Q*  of  coordinates 

A - a(aLcg(Zl)  + (3Log(Zm)  + vLog(Zs)) 

C,  - u.uoy ( i /m) 

C,  - U/LogU/s! 

(or  Cl  and 


i 

; 
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A’  - a (oLog (Z  ’ I ’ ) + flLoglZ’m’)  + vLog(Z’s')) 

C,'  - u,Log ( t * /m* ) 

C/  - u,Log(r/s’) 

for  Q’ . The  perceptual  difference  D between  those  two  patches  is 
defined  as  the  norm  of  the  vector  QQ’ 

0*  - (A  - A’)1  + (C,  - C,' ) J + (Cr  - C/)* 

Lie  want  to  minimize  D’  when  varying  I’,  all  other  variables  being 
constant 

d(0J)/dZ’  - d ( (A  - A’)l)/dZ7  - -2  (A  - A’MdA’/dZ*) 

It  is  interesting  at  this  point  to  note  that  minimizing  O’  is 
eciui valent  to  minimizing  (A  - A')1  which  is  the  brightness 
difference  between  the  two  patches.  A strai ghtforward  computation 
yields 

d(D,)/dZ’-(2a,/Z’)  (a+0+Y)  (c<Log(Z’  I ’ /Z  I ) +0Log (Z ’ m’  /Zm)  +YLog (Z*  s’  /Zs)  ) 

Keeping  only  first  order  terms  yields 

d(D*)/dZ’  = (2aJ/Z ) (a  + 0 + v)  (aAL/L  + <3Ar,/ri  + vAS/S) 

Where  AL  - (Z ’ I ’ - Zl)  and  two  other  similar  expressions  for  AM  and 
AS.  The  minimum  perceptual  difference  is  obtained  for 

oAl/L  + £An/n  + yAS/S  - 0 (A.l) 

Let  us  now  take  into  account  the  fact  that  patches  P and  P*  are 
monochromatic  of  wavelengths  A and  A + dA  and  radiances  R(A)  and 
ITU)  + dR  ( A)  . 1(A),  m ( A)  and  s(A)  being  the  cone  absorption  curves 

we  have 

L - I (A)  R (A)  AL  - ( (d  I (A)  /dA)  R (A)  + I (A)  (dR  (A) /dA) ) AA 
and  four  other  similar  expressions. 

Then 


AL/L  - ( (1/1 (A) ) (dl (A)/dA)  + (1/R (A) ) (dR (A) /dA) ) AA 
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and  two  similar  expressions. 

Equation  (A.l)  can  now  be  rewritten  as 

(a  + (3  + V)  (l/R(X))dR(X)/dX  + (a/  I (X) ) d I (X) /dX  + 

(/3/m  (X) ) dm  (X) /dX  + (y/s (X) ) ds (X) /dX  - 0 
or 

- (dR  (X)  /R  (X) ) - (1/a  + d + Y)  (ad  I (X)/l  (X)  + /3cJm(X)/m(X)  + 
Yds (X) /s  (X) ) 

This  differential  equation  is  readily  integrated  and  yields 
l/RIX)  - I (X)0'o*"*Tm(X)»'t,*‘,*rs(X)r/(,4‘’*r 
and  thus  the  relative  luminous  efficiency  function  predicted  by  the 
mode*  I i s 

V*  (X)  - k/R  (X)  - k I (X)  (X)  ‘>,“°*rs  (X)  T,““,*y 

uhercs  the  constant  k is  adjusted  to  make  the  maximum  equal  to  1. 
Constants  a,  p>,  Y are  adjusted  so  that  this  maximum  occurs  at  5S5nni. 
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OPTIMIZATION  CF  a,  u,.  u. 

Color  matches  have  a normal  cl  i s tr  i hut  i on  around  a given  color 
center  in  the  C1E  (x,  y , 0. 2 I og,0  IV) ) space  (Broun  and  Mac-Adam  ISAS 
[A]).  Given  that  just-noticeable  differences  are  about  three  times 
standard  deviations,  the  locus  of  colors  just  noticeably  different 
from  the  given  color  center  is  an  ellipsoid.  We  would  like  this 
ellipsoid  to  map  to  a sphere  of  unit  radius  in  the  (A,C,,C2)  space. 

In  order  to  do  that,  ue  first  assume  that  chromaticity  and 
brightness  errors  are  independent  which  allows  us  to  optimize 
independently  a and  (u|(u2). 

For  an  achromatic  color,  we  have  L=M«S  and  thus  A-aLog(L). 
Also,  if  the  model  is  calibrated  for  the  equal  energy  white,  then  we 
also  have  L-X-Y-Z  and  thus  A-aLog(Y).  This  implies 
(dA)2  » a2(dY/Y)2 

If  (dA)’-l  is  the  just  noticeable  difference  that  we  want 
a2 (dY/Y) 2 - (S3.13V9) (dY/li.53Y)2 

and  thus 

a - 22. 6 

Optimization  of  parameters  u,  and  u2  is  done  in  the  following  way. 
Uo  take  the  Mac-Adam  ellipses.  projections  of  the 
standard-deviations  ellipsoids  on  the  (x,y)  plane,  and  map  them  onto 
the  (C,,C2)  plane  of  the  model  (see  figure  B.l).  We  then  adjust  u, 
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and  u7  to  get  the  best  fit  with  circles  of  radius  1/3.  Table  1 
shows  the  results  for  u,-64  and  u2=10. 

TABLE  B. 1 


Color  center  Mac-Adam  ellipses  Model 


*0 

y0 

lOOOu 

lOOOv 

0 

3U 

3V 

o.ieo 

0.057 

0.85 

0.35 

62.5 

1.44 

0.72 

0.187 

0.118 

2.2 

0.55 

77.0 

0.72 

0.73 

0.253 

0.125 

2.5 

0.50 

55.5 

0.70 

0.70 

0.150 

0.680 

9.6 

2.3 

105 

1.70 

0.82 

0.131 

0.521 

4.7 

2.0 

112.5 

0.64 

0.74 

0.212 

0.550 

5.8 

2.3 

100.0 

0.97 

0.83 

0.258 

0.450 

5.0 

2.0 

92.0 

0.88 

0.81 

0.152 

0.365 

3.8 

1.9 

110.0 

0.63 

0.93 

0.280 

0.385 

4.0 

1.5 

75.5 

0.73 

0.69 

0.380 

0.498 

4.4 

1.2 

70.0 

1.72 

0.48 

0.160 

0.200 

2.1 

0.95 

104.0 

0.44 

0.82 

0.228 

0.250 

3.1 

0.90 

72.0 

0.62 

0.61 

0.305 

0.323 

2.3 

0.90 

58.0 

0.51 

0.49 

0.385 

0.393 

3.8 

1.6 

65.5 

0.95 

0.77 

0.472 

0.399 

3.2 

1.4 

51.0 

1.29 

0.73 

0.527 

0.350 

2.6 

1.3 

20.0 

1.25 

0.84 

0.475 

0.300 

2.9 

1.1 

28.5 

1.00 

0.84 

0.510 

0.236 

2.4 

1.2 

29.5 

0.69 

1.60 

0.596 

0.283 

2.6 

1.3 

13.0 

1.53 

1.50 

0.344 

0.284 

2.3 

0.90 

60.0 

0.45 

0.60 

0.390 

0.237 

2.5 

1.0 

47.0 

0.52 

0.93 

0.441 

0.198 

2.8 

0.95 

34.5 

0.64 

1.39 

0.278 

0.223 

2.4 

0.55 

57.5 

0.56 

0.43 

0.300 

0.163 

2.9 

0.60 

54.0 

0.62 

0.72 

0.365 

0.153 

3.6 

0.95 

40.0 

0.75 

1.60 

Fig.  B.l-  Mapping  of  MacAdam  ellipses  into  circles. 
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IMPORTANT  MATRICES 

Ue  had  to  solve  several  times  the  problem  of  finding  the  matrix 
representing  a change  of  primaries,  the  standard  primaries  being 
defined  by  the  C1E  X,  Y,  Z tristimulus  values.  This  situation  is 
depicted  on  figures  C.l  and  C.2 

CONES 
OUTPUTS 

U, 


PHOSPHORS 
PRIMARIES 

Fig,  C. 1-Computation  of  the  U,  matrix  from  matrices  E,  and  Dt. 
where  the  reference  uhite  g,  = 11,1.051,1.1441'  is  the  D6500  white. 

CONES 
OUTPUTS 


FILTERS 

PRIMARIES 


U 


C.  I . E 


Fig,  C.2-  Computation  of  the  U2  matrix  from  matrices  E2  and  D2. 
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where  the  reference  white  w?  - [1,1.019,1.174]*  is  the  CIE 
illuminant  C. 


Thus  we  have  11,-EJV1  (i— 1.2) . C 

),  and 

1 E,  are  normalized  so 

that: 

I - e£v  (i-1,2) 

(C. la) 

and 

I - D.w.  (i-1,2) 

(C. lb) 

Ei,  ( i-1  ,2) , stem 

from  E"*  given 

by  Stiies  ([61]  p.515) 

651  1260 

-112 

E-‘  - 

-438  1620 

123 

(C.  2) 

.708  0 

417 

D,  stems  from 

F,  such  that 

FV' 

1 has  for  column  vectors  the 

chromaticity  coordinates  x,  y, 

z of 

the  phosphors  lights 

. 

.628 

.343 

.155 

F.-*  - 

.345 

.585 

.066 

.027 

.072 

.779 

(C.  3) 


in  the  same  way  D,  stems  from  F?  such  that  F2*‘  has  for  column 
vectors  the  chromaticity  coordinates  x,  y,  z of  the  light  coming 
through  the  Wratten  filters  25,  58  and  47B^0)en  illuminated  by  the 
CIE  illuminant  C 


(C.  4) 


.6808 

.2425 

.1579 

Fr‘  » 

.3190 

.6923 

.0187 

.0002 

.0652 

. 8234 

The  normalization 

procedure  goes 

as  fol 

Let  us  rewrite  E’*  as 


rr..r,.r,J 


where  Ti  (i-1,2, 3)  is  the  i th  column  vector.  Then  E,"*  is  of  the 
form 


; 

: 
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E,-1  - t\Tl,*,T».XIT,]  - £-*X 

where 

X,  0 0 

X ■ 0 X,  0 

0 0 X, 

and  X,,  X,,  X,  are  three  parameters.  Then  equation  (C.la)  rewrites 
as 

w,  - E.-1!  - E“X  I 

thus 

XI  - e w. 
or 

(X|VX7,XB]t  ■ E Wj 

Matrices  D4,  EIt  D,  are  computed  exactly  the  same  way.  One  finally 
has: 


.2457 

.5840 

.0703' 

.1101 

.7525 

.1273 

.0132 

.0842 

.9025 

.3534 

.5102 

.0254 

.1245 

.8138 

.0515 

.0009 

.0502 

.9389, 

Another  important  matrix  is  the  matrix  which  achieves  the  separation 
of  brightness  and  chromatic  information 


aa 

a(J 

u4 

-u4 

u. 

0 

where  a-22.6,  u,»64  and  u,«10. 


1 

i 


APPENDIX  D 


CALIBRATION  PROCEDURES 

Uo  6aw  in  chapter  4 what  the  non  I inear  i t i 69  involved  in  the 
input  process  were  and  how  they  had  been  compensated  for.  The 
output  process  could  be  two  things:  either  display  the  image  on  the 
face  of  a television  monitor  (chapter  3)  or  display  the  image  row  by 
row  on  the  face  of  a CRT  which  was  used  to  expose  a piece  of  color 
f i I m . 

In  the  first  case  the  calibration  procedure  consisted  in  two 
stops.  First  the  white  (correspond i ng  to  equal  drive  signals  on  the 
tliroo  gunc)  was  precisely  adjusted  to  06530  using  a split-field 
color  comparator.  Then  the  nonlinearity  of  the  tube  was  estimated 
in  the  following  way.  A digital  step-wedge  pattern  consisting  of 
density  steps  between  2.5  and  8 by  steps  of  .5  (thus  correspond i ng 
to  digital  intensities  between  2T2.5  and  2T8  by  steps  of  2T.5)  was 
displayed  on  the  face  of  the  television  monitor  and  the  luminance  of 
ovory  step  was  measured  with  a Tektronix  J16  photometer  connected  to 
a JG503  luminance  probe.  The  graph  I urn i nance/ i ntens i ty  turned  out 
to  bo  an  exact  straight  line  of  6lope  2.5  on  a Log-Log  scale  thus 
indicating  a gamma  of  2.5  for  the  tube.  This  was  compensated  for  by 
table  lookup  in  the  Comtal  before  conversion  through  the  8-bit  DACS. 

In  the  second  case  the  method  was  pretty  much  the  same  except 
for  the  complication  introduced  by  the  film.  A step-wedge  pattern 
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hob  used  to  expose  a piece  of  color  film  through  three  filters.  The 
film  was  then  processed  and  the  negative  printed  for  an  exact 
dons i tometr i c grey  on  the  fifth  step.  The  D-LogE  curves  were  then 
measured  for  the  three  colors  red,  green  and  btue.  A typical  set  of 
such  curves  is  shown  in  figure  0.1.  The  corresponding 
non  I i near  i t i es  (CRT  and  film)  were  again  compensated  for  by  table 
lookup  before  conversion  through  the  11-bit  DAC.  Examples  of  an 
uncompensated  and  properly  compensated  color-wedge  are  shown  in 
figure  D.2.  For  the  compensated  one,  the  three  D-LogE  curves  are 
straight  lines  of  slope  1,  thus  guaranteeing  that  photographic 
intensities  correspond  closely  to  digital  intensities. 

The  procedure  for  printing  on  photographic  paper  the  images 
shown  in  this  report  was  then  the  following.  Every  time  we  exposed 
color  film  on  the  CRT,  we  also  exposed  a piece  of  film  with  a wedge. 
The  films  were  then  developed.  The  wedge  was  printed  for  a grey  so 
that  the  three  D-LogE  curves  were  straight  lines  of  slope  1.  Then 
the  whole  batch  of  negatives  was  printed  exactly  the  same  way  as  the 
wodgo  thus  reducing  to  a minimum  distortions  introduced  by  film 
processing  in  the  photo- lab. 


(a) 


(b) 


Fig.  D.2-  (a)  uncompensated  and  (b)  compensated  color  stepwedges. 
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DESCRIPTION  OF  THE  PSYCHOPHYSICAL  EXPERIMENT 
C.l  The  experiment 

The  experiment  was  intended  to  test  the.  linearity  of  spatial 
neural  interaction  in  the  chroma t i c i ty  channels  C,  and  C7  and  answer 
the  C|uostion  of  whether  this  interaction  was  charac  ter  i zed  by  an 
attenuation  of  low  spatial  f requenc i es.  We  excited  one  channel  at  a 
timo  with  a pure  chromatic  pattern  of  the  form 

C,  - c,  + k,  (sin(2nfx)  + asin(6nfx))  (E.la) 

C7  ■*  c7  (E.  lb) 

and 

C.  - c,  (E.lc) 

C,  » c,  + k, (s i n (2rtfx)  + asin(Snfx))  (E.ld) 

following  the  method  pioneered  by  Baudelaire.  The  brightness 
channel  was  constant  in  both  cases  (A=a) . The  values  of  parameters 
u,  and  u,  controlled  the  position  of  the  average  point  in  the 
(C..C,)  plane  (see  figure).  After  mapping  through  the  inverse  of 
tho  model  of  figure  2.3,  that  is  to  say  through  followed  by 
exponentiation  on  each  component  and  U"1,  those  patterns  were 
presented  to  a set  of  five  observers  of  both  sexes  on  the  face  of  a 
COMTAL  where  they  sustained  a field  of  view  of  7 degrees. 

Wo  experimented  with  four  spatial  frequencies 
fj  - 0.142  cycles/degree 


f7  - 0.284  cycles/degree 
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■ 


f*  - 0.568  cycles/degree 
ik  - 1.136  eye  I es/degree 

In  all  cases,  all  subjects  declared  that  the  brightness  was 
uniform  accross  the  pattern  thus  indicating  that  the  brightness 
do  fined  by  the  model  is  in  good  agreement  with  perceptual 
br i ghtnese. 

Two  central  bands  whose  subjective  appearance  is  controlled  by 
simultaneous  color  contrast  were  indicated  by  markers.  The  subjects 
warn  asked  to  focus  their  gaze  at  the  center  of  the  pattern  and 
immediately  report  verbally  their  judgement  by  one  of  the  following 
answer os 

R (right):  The  band  to  the  right  is  redder 
L (left):  The  band  to  the  left  is  redder 
S (same):  The  two  bands  are  the  same  color 
for  the  C, -channel  and 

R (right):  The  band  to  the  right  is  yellower 
L (left):  The  band  to  the  left  is  yellower 
S (same):  The  two  bands  are  the  same  color 
for  the  C,-channe I . 

For  each  spatial  frequency  and  every  channel,  the  parameter  a 
was  varied  from  1 to  .1  by  steps  of  .1.  The  contrast  was  kept 
constant  by  adjusting  the  parameter  k in  equations  (E.la)  and 
(E.ld).  For  o-l  a 6trong  simultaneous  color  contrast  illusion  was 
produced  except  for  frequency  f4  in  the  C?-channe I . For  a-.l,  the 
fast  varying  component  disappears  completely  ir  equations  (E.la)  and 
(E.ld),  thus  producing  what  may  be  called  an  over-compensation  of 
the  color  illusion.  The  idea  of  the  experiment  is  to  estimate  the 
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i ntormedi  a te  value  of  a which  produces  target  bands  of  same  color. 
This  value  of  a is  then  considered  as  indicating  the  value  of  the 
relative  amplification  of  frequencies  f and  3f  for  the  particular 
chromatic  channel  tested.  Each  set  of  ten  stimuli  was  shown  twice 
to  every  subject,  the  order  in  the  set  being  reversed  from  one 
presentation  to  the  next. 

E. 2 The  resu l ts 

The  values  o,  (i-1,2)  for  which  the  target  bands  appear  to  be 
the  same  color  are  defined  as  the  point  of  subjective  equality  (PSE) 
for  the  i th  channel.  For  every  channel,  every  observer  and  every 
frequency,  the  parameter  a,  was  estimated  by  taking  the  average  of 
the  two  presentations.  For  each  presentation,  the  PSE  was  taken  as 
the  mean  of  the  transition  values  T*  and  T"  of  a,  where  the 
r.uh  joc  1 1 ve  judgement  switched  from  R to  S and  f'-om  S to  L 
respec t i ve I y , as  shown  on  the  following  example  based  on  the  typical 
data  of  table  Is 

average  of  T*  - <T*>  - 0.50 
average  of  T*  - <T’>  - 0.43 

interval  of  uncertainty  IU  - <T*>  - <T'>  - 0.10 
point  of  subjective  equality  PSE  - (<T*>  + «.T'>)/2  - 0.45 
Tables  2 and  3 contain  the  experimental  results  for  the  C,-channel, 
tab  I os  4 and  5 contain  the  experimental  results  for  tne  C,-channe  I . 
Wo  averaged  those  results  over  the  five  subjects  and  obtained  the 
following  means  and  standard-deviations 


A 


in 


C,-channe I : 


point  of  subjective  equality 
frequency  f 
mean 
s. dev. 


f,  f.  u 

0.50  0.54  0.50  0.49 
0.05  0.04  0.05  0.11 


interval  of  uncertainty 
frequency 
mean 
s. dev. 


f.  1,  u ** 


0.13  0. 15  0.14  0.15 
0.05  0.06  0.05  0.09 


C7-channe I : 

point  of  subjective  equality 

frequency  f,  f7  fa 

mean  0.61  0.59  0.63 

s.dev  0.05  0.12  0.12 

Interval  of  uncertainty 
frequency 
mean 
s. dev. 

E.3  Conclusions 

Ue  concluded  from  these  experimental  results  that  the  point  of 
subjective  equality  was  about  constant  over  the  range  of  frequencies 
studied,  equal  to  0.50  for  the  C,-channel  and  0.60  for  the 
C7-channe I . The  yellow-blue  channel  has  a lower  contrast 

sensitivity  than  the  red-green  chaqnel  combined  with  a lower  visual 
acu i ty . 


f,  f?  f a 
0.10  0.14  0.10 
0.05  0.04  0.03 


r 
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Indeed,  the  results  for  frequency  ft  are  not  shown  for  this 
channel  because  the  illusion  was  seen  in  reverse  by  four  out  of  the 
f i vo  observers  and  seen  much  weaker  by  the  fifth  one.  As  we 
mentioned  in  chapter  3,  we  used  this  fact  to  estimate  the  position 
of  the  peak  frequency  of  the  frequency  response  H,. 
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TABLE  E. 2 

POINT  OF  SUBJECTIVE  EQUALITY 
(C, -CHANNEL) 

Frequenc i es 


f. 

U 

U 

Suljjoct  1 

.450 

.525 

.500 

.500 

Subject  2 

.575 

.575 

.575 

.500 

Suljjoct  3 

.525 

.500 

.525 

.600 

Subject  4 

.500 

.500 

.475 

.475 

Subject  S 

.450 

.530 

.425 

.300 

TABLE  E. 3 

INTERVAL  OF  UNCERTAINTY 
(C, -CHANNEL) 

Frequenc i es 


U 

f. 

f. 

U 

Sub joc t 

1 

.100 

.250 

.203 

.200 

Si  ib  j ec  t 

2 

.150 

.150 

.153 

0 

Sub  joc t 

3 

.053 

.130 

.053 

.200 

Subject 

4 

.150 

.100 

.150 

.250 

Sub joc t 

5 

.203 

.230 

.150 

.130 
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TABLE  E.4 

POINT  OF  SUBJECTIVE  EQUALITY 
(^-CHANNEL) 


Frequenc i es 

U U f. 


Sub  j l!C  t 

1 

.025 

.075 

.075 

Sub  j ec  t 

2 

.550 

.400 

.500 

Subject 

3 

.053 

.075 

.800 

Sub  j oc  t 

4 

.075 

.700 

.700 

Subject 

5 

.550 

.500 

.475 

TABLE  E.5 

INTERVAL  OF  UNCERTAINTY 
(C,-CHANNEL) 

Frequenc i ea 


U 

f. 

Sub jnc  t 

1 

.050 

.150 

.150 

Sub  joc t 

2 

.103 

.208 

.100 

Sub  ject 

3 

.103 

.150 

.103 

Sub ject 

4 

.050 

.100 

.100 

Sub  j ec  t 

5 

.200 

.100 

.150 
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